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1  Executive  Summary 

This  report  summarizes  the  work  performed  during  this  Phase  II  Small  Business  Innovation  Research 
(SBIR)  project.  All  objectives  stated  in  the  Phase  II  proposal  were  accomplished.  Furthermore,  the 
SHAI  and  the  primary  sub-contractor  Klein  Associates,  Inc,  engaged  in  significant  interaction  with 
LSOs,  which  was  incredibly  important  for  this  effort. 

2  Problem  Statement 

The  Aircraft  Carrier,  or  CV,  landing  environment  is  an  extremely  complex  one.  In  addition  to  operating 
what  may  be  termed  as  an  extremely  busy  airport,  CV  landing  operations  are  affected  by  a  number  of 
variables  not  associated  with  a  normal  aerodrome.  Of  these,  the  most  critical  are  fleet  tactical 
considerations,  flight  deck  space  constraints,  CV  maneuvering  space  (sea  room),  flight  deck  motion 
(pitch  and  roll),  continuous  mechanical  preparations,  resetting  arresting  gear  and  optical  landing  system 
between  each  landing,  airborne  aircraft  fuel  status  and  management  of  aerial  refueling  assets,  aircraft 
ordinance,  minimal  use  of  navigation,  communications  and  radar  emissions  as  in  EMCON  operations 
and,  above  all,  time  constraints. 

Safe  and  efficient  control  of  this  environment  requires  following  a  strict  chain  of  command  and  adhering 
to  a  set  of  standard  operating  procedures.  The  chain  of  command  follows  from  the  CV  Captain,  through 
the  Air  Operations  Officer  (Airops),  below  the  flight  deck,  and  the  Air  Officer  (Air  Boss),  in  the  tower, 
to  the  Landing  Signal  Officer  (LSO),  stationed  at  the  stem  of  the  ship  next  to  the  landing  area.  The 
Captain  is  ultimately  responsible  for  the  entire  operation  of  his  ship.  The  Airops  Officer  is  responsible 
for  aircraft  outside  a  five  mile  radius  to  a  distance  of  twenty  miles  from  the  ship,  as  well  as  managing 
airborne  fuel/aerial  refueling  and  aircraft  status;  and  providing  surveillance  and  precision  radar  guidance 
to  the  pilots  for  both  night  and  low  visibility  approaches  and  landings.  The  Air  Boss  is  responsible  for 
aircraft  within  five  nautical  miles  as  well  as  all  flight  deck  preparations,  aircraft  handling  on  the  flight 
deck,  and  final  landing  clearance.  The  LSO  is  responsible  for  the  aircraft’s  final  approach  and  landing. 

During  the  last  60  seconds,  the  cognitive  demands,  namely  the  critical  decisions  and  judgments,  increase 
quickly  until  a  decision  to  wave  off,  or  not,  is  made.  Day  landings  with  good  weather  are  ideal 
conditions,  but  unfortunately  not  all  days,  or  nights,  are  like  that.  Often  times  the  ship  is  heaving  10  ft. 
up  and  10  ft.  down,  making  a  20  ft.  displacement  from  a  level  deck.  In  addition,  it  is  often  difficult  to  see 
the  aircraft  approach  during  night  operations,  and  impossible  to  see  during  stormy  conditions  at  night. 
The  LSOs  must  rely  on  auditory  cues  and  the  equipment  at  the  LSO  station  to  assist  their  decision¬ 
making.  For  this  project,  we  were  tasked  to  design  a  decision  support  tool  that  will  assist  LSO  decision¬ 
making  and  hopefiilly  increase  the  amount  of  time  to  make  a  wave-off  decision,  which  is  usually  about 
0.5-4  seconds.  The  focus  of  the  decision  support  tool  was  to  provide  pilot  trending  information  along 
with  key  oscillation  deviations  so  that  the  LSO  could  improve  both  safety  and  efficiency  of  recoveries. 
We  feel  that  we  have  done  this  with  our  current  interface,  and  have  taken  the  assignment  one  step  further 
in  providing  predictions  for  aircraft  and  deck  position,  two  key  oscillations  during  flight  operations.  This 
report  will  describe  the  LSO  environment,  the  approach  we  took  to  investigating  pilot  trending  and  key 
oscillations,  and  the  development  of  a  decision  support  interface  to  be  implemented  in  the  VISUAL 
system,  a  larger  information  system  scheduled  to  be  included  on  the  LSO  platform. 

2.1  Day  Operations 

Daytime  landing  operations  are  referred  to  as  Case  I  recoveries.  With  Case  I  recoveries,  the  aircraft  fly 
by  the  starboard  side  of  the  ship  (downwind)  and  perform  a  break  once  they  are  past  the  bow.  The 
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aircraft  continues  the  turn  to  fly  upwind  at  about  1  mile  off  the  port  side  into  a  final,  gradual  180-degree 
turn  and  arrives  at  a  point  1/2  to  3/4  of  a  mile  astern  the  ship.  The  landing  area  is  then  prepared  for 
recovery,  which  is  normally  within  a  45  to  60  second  separation  interval  from  the  aircraft  ahead.  The 
Senior  LSO  or  one  of  the  two  Airing  Staff  LSOs  “waves”  the  approaching  aircraft  from  the  LSO 
Platform. 


Several  other  individuals  assist  these  LSOs;  some  are  LSO’s  and  some  are  enlisted  personnel.  Normally 
the  most  senior  LSO  will  be  the  “backup”  or  supervising  LSO  while  a  qualified,  but  more  junior 
individual  will  be  the  “controlling”  LSO.  A  third  individual  will  copy  shorthand  comments  into  the 
LSO’s  Grade  Log  for  use  after  the  recovery  in  debriefing  each  pilot  regarding  his  approach. 

Occasionally,  other  less  experienced  LSO’s  will  observe  the  recovery  for  training.  Information  available 
to  the  LSO  on  the  platform  is  provided  in  console  displays  including:  Wind-over-the  deck;  optical 
landing  system  status  and  lamp  intensity  indicators  and  controls;  clear  deck/foul  deck  (green  light/red 
light)  indicator;  deck  motion  indicators;  and  a  CRT  display  from  a  fixed  centerline  video  camera  view  up 
the  glide  path  with  stabilized  crosshairs.  This  final  source  of  information  is  the  pilot  landing  aid 
television  (PLAT)  from  which  a  videotape  is  recorded  for  later  reference  as  well  as  mishap  investigation. 
The  LSO's  communications  resources  consist  of  two  radio  sets  (under  his  control),  access  to  two  radar 
controller  radio  circuits,  a  ship’s  telephone,  several  sound-powered  phone  circuits  and  one  or  two 
enlisted  deck  personnel  in  constant  communication  with  other  critical  recovery  operations  workstation 
operators.  The  Captain  and  Air  Boss  can  also  communicate  with  the  LSOs  via  their  individual  public 
address  systems  from  the  ship’s  tower. 

The  deck  preparation  includes  stowing  loose  equipment,  moving  personnel  from  their  launch  to  their 
recovery  positions,  moving  aircraft  from  the  landing  area,  and  retracting  the  arresting  cable. 
Simultaneously,  the  optical  landing  system  is  reset  for  the  next  type  of  aircraft  approaching,  and  an 
enlistee  on  the  LSO  platform  is  also  checking  that  the  approaching  aircraft  is  in  the  proper  configuration 
for  landing  (landing  gear  locked  down,  tail-hook  down  and  flaps/slats  extended).  As  the  aircraft 
approaches,  the  LSO  assesses  the  pilot’s  response  to  deviations  from  on-speed,  centerline,  and 
glideslope.  None  of  these  factors  remain  constant  as  the  landing  area  is  angled  and  constantly  moving  to 
the  right  of  the  aircraft’s  flight  path  as  the  ship  moves  forward.  The  aerodynamics  of  the  flight  path  is 
affected  by  the  wind  over  the  deck  as  it  flows  down  off  the  stem  then  rises  off  the  water  1/4  to  1/2  mile 
behind  the  ship  at  varying  intensities.  This  effect  is  associated  with  a  variance  of  aircraft  airspeed  as  the 
pilot  constantly  corrects  for  these  and  his  own  induced  deviations.  At  a  point  in  the  aircraft’s  approach, 
usually  within  0.5  -  4  seconds  of  recovery,  the  LSO  must  then  determine  whether  the  aircraft  position  and 
speed  is  stable  and  safe  enough  to  complete  a  landing  to  a  touchdown  area  approximately  20  feet  wide  by 

7 


Stottler  Henke  Associates,  Inc.  N68335-98-C-0027 

200  feet  long,  or  signal  the  pilot  to  execute  a  wave-off.  If  the  deck  is  still  being  prepared  for  the  landing, 
the  LSO  must  judge  whether  or  not  the  deck  can  be  cleared  in  time  for  a  safe  recovery.  At  the 
completion  of  a  wave-off  or  landing  the  LSO  will  again  begin  assessing  the  next  aircraft  while  verbally 
evaluating  the  last  approach  to  his  assistant. 

2.2  Night  Operations 

Night  operations  fall  into  the  Case  II  or  Case  m  recovery  classification.  Case  II  recoveries  follow  the 
same  approach  pattern  as  Case  I  (daytime)  operations,  while  Case  in  recoveries  have  the  aircraft 
marshalling  approximately  20  miles  out  and  they  come  straight  in  to  the  carrier.  Case  El  recoveries 
encompass  situations  where  visibility  is  very  low  and/or  weather  conditions  are  poor.  These  are  the  most 
difficult  recoveries.  In  all  three  types  of  recoveries,  the  approach  patterns  are  identical  from 
approximately  %  of  a  mile  from  the  ramp  on  in. 

In  Case  El  recoveries,  the  marshalling  area  is,  as  mentioned,  20  miles  out.  The  aircraft  then  follow  an 
instrument  approach  procedure  to  arrive  at  a  point  10  miles  behind  the  ship,  then  receive  either  verbal 
Precision  Approach  Radar  guidance,  a  precise  Instrument  Landing  System  display  in  the  cockpit,  or  are 
automatically  controlled  to  touchdown  by  coupling  the  aircraft’s  autopilot  to  the  ship’s  Automatic 
Carrier  Landing  System.  These  approaches  are  generally  more  stable  at  the  start  due  to  the  long  straight- 
in  flight  path  versus  a  turning  approach  to  a  short  wings-level  final  as  in  Case  I  and  E.  The  LSO  is  also 
presented  with  repeaters  of  the  aircraft’s  performance  via  the  information  directly  accessed  from  the 
precision  radar.  This  is  presented  on  a  collector  lens  directly  in  front  of  the  LSO  in  the  form  of  a  Heads 
Up  Display  (HUD).  The  LSO  can  use  this  information  to  anticipate  errors  from  these  visual  and  aural 
cues.  The  HUD  is  of  greatest  use  when  meteorological  conditions  restrict  the  LSOs  normal  ability  to 
watch  the  aircraft  approach  visually.  It  is  under  these  visually  restricted  conditions  (darkness  on  the 
flight  deck,  darkness  looking  astern  the  ship,  and  the  associated  loss  of  depth  perception  at  night,  etc.), 
that  the  LSOs  face  their  greatest  challenges. 

2.3  LSO  Challenges 

The  Landing  Signal  Officer  faces  many  challenges.  Although  LSOs  primary  concern  remains  safety,  the 
ship  is  under  significant  pressure  to  maintain  an  extremely  rapid  recoveiy  rate.  The  average  recovery 
rates  necessitate  small  intervals  (45  to  60  seconds)  between  each  landing,  and  there  is  constant  pressure 
to  not  wave-off  unless  absolutely  necessary.  Additionally,  due  to  EMCON  constraints,  the  LSO 
minimizes  use  of  the  radio.  Consequently,  LSOs  have  little  time  to  make  the  wave-off  decision,  are  often 
forced  to  wait  until  the  last  possible  second  to  make  the  final  decision,  and  often  may  not  have  a  good 
definition  of  what  the  last  second  is.  Many  of  the  issues  an  LSO  must  address  are  juggled  between  a 
series  of  trade-offs:  safety  vs.  effectiveness;  wave-off  a  bad  pass  vs.  getting  the  pilot  on  board,  etc. 

During  day  operations  in  clear  weather  conditions,  the  approach  radar  is  not  normally  used  and  therefore 
much  of  the  information  that  would  be  available  from  this  equipment  (e.g.,  speed,  actual  rate-of  descent, 
distance,  etc.)  is  not  directly  available.  During  these  approaches,  the  LSOs  visually  ascertain  aircraft 
attitude,  from  which  air  speed  can  be  inferred.  As  the  LSO  visually  monitors  the  aircraft  for  proper 
glideslope  and  line-up,  current  throttle  setting  can  also  be  inferred  by  listening  to  the  engine.  The  LSO  is 
also  adept  at  predicting  what  the  pilots  next  move  might  be  based  on  the  dynamics  of  the  flight  path. 

This  is  an  important  parameter  because  to  reject  the  landing,  the  pilot  must  go  to  full-throttle,  which  may 
take  up  to  6.5  seconds  to  take  effect,  depending  on  the  current  throttle  settings  and  aircraft  type.  This  is  a 
considerable  period  of  time,  given  the  split-second  decision-making  performed  by  the  LSO. 
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In  addition  to  recognizing  particular  aircraft  and  pilot  model  (different  for  each  pilot  and  aircraft  type), 
the  LSO  must  also  consider  status  of  the  deck  condition  and  crew,  specifically  whether  the  deck  is  clear 
and  crew  is  ready  for  recovery.  If  the  deck  is  foul,  the  Air  Boss  will  only  notify  the  LSO  if  it  is 
absolutely  sure  that  the  deck  will  not  be  ready  in  time  and  will  call  a  no  chance  wave-off.  Normally  the 
LSO  will  wait  until  the  last  possible  moment  to  wave  the  aircraft  off,  since  the  deck  may  be  ready  just  at 
that  last  possible  moment.  Knowledge  of  the  deck  crew,  their  capabilities,  and  speed  of  preparation  are 
used  by  the  LSO  to  better  estimate  the  likelihood  that  the  deck  will  be  clear.  The  pitch  and  roll  of  the 
deck  must  also  be  considered,  as  these  factors  influence  the  pilot’s  ability  to  perceive  correct  line-up  and 
glideslope. 

In  addition  to  having  little  precise  information,  an  ill-defined  point  of  wave-off,  working  under  extremely 
high  time  pressure  and  unpredictable  environmental  conditions,  and  having  various  pilot  approach 
aspects  to  simultaneously  consider,  the  LSO  must  estimate  the  wave-off  window  as  it  changes  with  the 
current  conditions.  The  latest  point  of  the  wave-off  window  is  defined  as  the  point  at  which  the  LSO  can 
wave  the  pilot  off  such  that  he  will  pass  at  a  particular  minimum  height  above  the  flight  deck.  Obviously 
this  point  shifts  with  the  conditions  and  thus  must  be  estimated  by  the  LSO. 

Finally,  the  LSO  attempts  to  exercises  as  little  control  of  the  aircraft  as  possible,  unless  required  by 
safety  considerations.  Otherwise,  they  can  be  more  of  a  distraction  to  the  pilot  and  potentially  make 
matters  worse.  Being  active  Navy  pilots  themselves,  LSOs  have  learned  that  having  too  much  of  an  LSO 
in  the  cockpit  can  be  very  confusing.  Along  these  lines,  LSOs  have  also  learned  that  it  complicates 
things  when  they  try  to  give  pilots  specific  control  instructions  along  more  than  one  dimension  at  a  time 
(e.g.,  passing  on  line-up  and  glide  slope  corrections  simultaneously).  They  recognize  that  they  have  to 
prioritize  the  correction  information  they  pass  on,  giving  the  pilot  the  most  critical  corrections  first, 
followed  later  by  additional  less  critical  corrections. 

The  LSOs  and  pilots  situation  is  greatly  different  at  night.  Night  recoveries  are  considered  much  more 
dangerous  than  daylight  operations.  During  night  conditions  LSOs  use  the  approach  radar,  which  gives 
them  more  accurate  distance,  azimuth,  and  bearing  information  such  that  deviations  from  glideslope  and 
line-up  can  be  readily  calculated  and  displayed  in  the  cockpit  and  repeated  on  the  LSO  Heads  Up 
Display.  Unfortunately,  the  accuracy  of  the  LSOs  information  tends  to  exaggerate  the  trends  that  the 
LSO  is  monitoring  when  compared  to  daylight  operations.  Although  more  accurate  deviation 
information  is  available  to  the  LSO  at  night,  he  still  relies  heavily  on  visual  perceptions  of 
airspeed/attitude,  centerline  displacement,  and  glideslope  control  and  is  therefore  challenged  with 
integrating  multiple  information  resources. 

As  a  further  complication,  every  pass  and  recovery  of  every  pilot  is  graded.  So,  in  addition  to  controlling 
the  approach,  the  Controlling  and  Backup  LSOs  are  also  yelling  out  their  observations  for  the  Logging 
LSO  to  record  for  later  use  in  the  pilot  debrief. 

3  Approach 

The  goal  of  this  project,  in  addition  to  developing  display  and  platform  recommendations,  was  to  employ 
a  combination  of  intelligent  system  techniques  from  the  field  of  Artificial  Intelligence  (AI)  and  decision- 
centered  techniques  from  the  field  of  cognitive  psychology.  SHAI,  the  prime,  has  extensive  AI  expertise, 
while  Klein  Associates  brings  expertise  in  cognitive  psychology  and  decision-centered  design.  The 
following  sections  will  describe  each  method  used  in  this  project  and  they  include:  decision-centered 
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design,  CTA  tools  (Critical  Decision  Method  and  Knowledge  Audit),  case-based  reasoning  (CBR),  Fuzzy 
Logic,  Neural  Networks,  and  Neural  Networks  Based  Fuzzy  Inference  System. 

Working  with  Landing  Signal  Officers  to  understand  how  they  approach  their  tasks,  and  how  best  to 
support  or  enhance  their  performance  has  been  a  central  theme  of  Klein  Associates  research  for  the  past 
three  years.  For  the  Phase  I  work  (Stottler  &  Thordsen,  1997),  Klein  Associates  and  SHAI  were  tasked  to 
learn  the  overall  task  of  recovering  aircraft  aboard  U.S.  aircraft  carriers.  It  was  a  fairly  broad  approach 
that  did  not  restrict  itself  to  only  the  LSOs  on  the  LSO  platform,  but  also  took  into  consideration  the  roles 
of  enlisted  personnel  on  the  platform  as  well  as  those  individuals  in  the  tower  and  air  operations  (Air 
Ops).  The  goal  of  the  Phase  I  research  was  to  investigate  the  feasibility  and  usefulness  of  combining 
these  cognitive  and  AI  approaches.  The  Phase  I  resulted  in  display  recommendations  for  the  Controlling 
LSO  (the  individual  who  actually  controls  the  aircraft  through  its  final  approach  to  the  recovery). 

For  the  Phase  n,  we  concentrated  specifically  on  pilot  trending  and  advanced  decision  aid  considerations. 
Our  objective  in  this  Phase  II  effort  was  to  identify  the  critical  pilot  trends  that  an  LSO  must  contend 
with,  and  to  build  an  advanced  decision-support  interface  that  supports  the  split-second  decisions  and 
perceptual  workload  pf  the  LSO.  We  identified  three  dynamic  aspects  of  the  approach:  the  aircraft,  the 
deck,  and  the  individual  pilot  (preferences,  habits,  and  trends). 

Our  work  is  consistently  guided  by  an  approach  to  system  design  we  have  termed  decision-centered.  In 
the  sections  that  follow,  we  will  provide  descriptions  of  decision-centered  design  and  our  approach  to 
CTA  knowledge  elicitation  and  representation.  Following  that,  we  will  introduce  multiple  AI  techniques 
that  were  used  to  assimilate  data  collected  from  CTA  knowledge  elicitation.  We  turn  then  to  a  discussion 
of  results  of  data  collection  as  they  relate  to  the  problems  and  issues  that  surround  development  of  a 
sturdy  and  flexible  interface  for  the  LSO  operator. 

3.1  Decision-Centered  Design 

A  decision-centered  approach  (Kaempf  &  Miller,  1993;  Klein,  1993;  Klein,  Kaempf,  Wolf,  Thordsen, 
Miller,  1997)  to  design  involves  using  Cognitive  Task  Analysis  (CTA)  to  identify  the  critical  decisions, 
judgments,  and  cognitive  elements  of  the  task  and  then  applying  this  information  to  any  of  a  number  of 
purposes.  A  decision-centered  approach  is  best  understood  when  presented  in  contrast  to  data-centered 
and  system-centered  approaches.  The  distinction's  an  important  one  for  understanding  consequences  for 
design  and  operator  performance,  and  is  described  below. 

Many  approaches  to  system  design  have  been  driven  by  the  information-processing  power  of  emerging 
new  technologies.  The  capabilities  of  these  high  power  systems  permit  access  to  vast  amounts  of  raw 
data.  Any  or  all  of  these  data  may  be  made  available  to,  and  sometimes  even  imposed  on,  the 
users/operators.  The  display  and  control  design  is  data-centered  and  technology  driven.  This  is  not 
surprising  given  the  great  temptation,  in  the  face  of  extraordinary  memory  capacity  and  operating  power, 
to  provide  as  much  information  to  the  individual  as  is  technologically  possible.  The  problem  with  a  data- 
centered  approach  is  that  it  does  not  take  into  account  the  shifting  contexts  in  which  many  users  function, 
it  totally  ignores  what  the  user  needs,  when  he  or  she  needs  it,  or  how  it  should  be  represented.  We  have 
seen  many  situations  where  individuals  (e.g.,  pilots  and  other  operators  of  complex  systems)  begin  by 
turning  off  various  support  and  warning  systems  because  the  operators  say  they  are  so  distracting.  Vast 
amounts  of  generic  data  can  interfere,  can  result  in  information  overload,  and  can  force  the  operator  to 
use  valuable  time  sorting  through  data  that  may  be  important  in  other  situations,  in  order  to  find  the  one 
or  two  pieces  that  are  important  in  this  situation.  More  is  not  always  better.  In  fact,  more  is  dangerous. 
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A  second  approach  can  be  termed  system-centered.  The  human  factors  community  has  recognized  the 
difficulties  of  design  that  is  technology  driven.  They  understand  that  the  information  requirements  of  the 
user  are  of  utmost  importance  if  a  complex  system,  which  includes  both  the  individual  and  the 
equipment,  is  to  perform  effectively.  One  approach  to  this  has  been  system-centered  designs,  where  data 
are  organized  and  presented  within  the  context  of  the  various  electronic  or  mechanical  sub-systems.  For 
example,  on  an  aircraft  these  systems  might  include  fuel,  weapons  status,  hydraulics,  and  navigation.  For 
the  LSOs  these  might  include  the  aircraft,  the  deck,  the  weather,  etc.  Overall,  the  system-centered 
approach  has  allowed  marked  improvements  in  interface  design.  It  presents  data  organized  in  a  way  that 
helps  the  user  understand  the  status  of  various  systems  and  equipment.  In  the  cockpit  example, 
information  for  aircraft  control,  navigation,  fuel  management,  ordnance  management,  and  tactical  and 
mission  data  are  all  available  to  the  pilot  in  various  forms  and  displays.  However,  the  system-centered 
approach  does  not  recognize  that  the  data  it  provides  generally  plays  a  supporting  role  to  the  cognitive 
processes  that  the  users  require  in  order  to  achieve  their  missions.  In  other  words,  while  the  system 
provides  a  variety  of  data  elements,  it  is  left  to  the  user  to  synthesize  the  data  to  answer  a  particular 
question,  or  to  fit  data  to  the  needs  of  the  current  situation.  For  example,  a  system  display  that  shows  a 
pilot  the  status  of  the  fuel  system  is  very  helpful,  however,  this  information  is  usually  only  a  sub-part  of 
the  overall  information  the  pilot  may  need.  More  often  than  not,  fuel  status  is  related  to  other  factors  such 
as  distance  and  time,  and  in  the  aviation  domain,  wind  and  speed  emerge  as  important  factors  as  well. 

The  integration  of  these  factors  would  be  more  useful,  perhaps,  than  just  a  fuel  status  display. 

It  is  our  view  that  for  the  user  to  think  and  act  effectively,  that  data  presentation  has  to  take  into  account 
the  context  of  the  individual’s  decision  making,  rather  than  being  data-  or  system-centered.  In  effect,  it 
needs  to  be  decision-centered.  To  achieve  this,  the  information  must  be  presented  in  a  functionally 
meaningful  way  the  information  must  be  framed  by  the  nature  of  the  critical  decisions  and  judgments 
(i.e.,  decision  requirements)  within  a  particular  context  or  situation,  and  made  available  to  system  users 
in  ways  that  support  thinking  and  action. 

A  decision-centered  approach,  as  the  name  implies,  anchors  the  design  around  the  decisions  that  will 
confront  the  user  who  is  involved  with  the  systems  tasks.  A  decision-centered  approach  can  be  viewed  as 
a  variation  of  Cognitive  Systems  Engineering,  one  in  which  decision  requirements  (the  most  critical  and 
difficult  decisions  and  judgments)  provide  the  foundation  for  the  generation  of  the  design  principles  and 
recommendations.  Decision-centered  design  is  beneficial  in  any  domain  that  involves  interaction  of 
human  and  smart  machines  to  accomplish  complex  tasks.  However,  in  domains  that  involve  extreme  time 
pressure  and  risk,  as  in  the  LSO  domain,  decision-centered  design  is  critically  important  to  optimal 
performance  and  avoidance  of  mistakes. 

3.2  Cognitive  Task  Analysis 

Our  approach  to  decision-centered  design  is  grounded  in  the  use  of  Cognitive  Task  Analysis  (CTA)  tools 
and  techniques  that  are  associated  with  critical  event  and  critical  decision  methodologies.  The  key 
element  of  these  approaches  is  that  they  derive  their  data  from  a  combination  of  observations  of  and 
interviews  about  real-world  decisions  and  events.  For  this  project,  the  CTA  helped  us  identify  and 
document  cognitive  elements  (i.e.,  critical  decisions  and  judgments)  of  the  LSO  task,  so  that  we  could 
incorporate  these  critical  elements  into  the  design  of  a  flexible  and  high-powered  interface. 

CTA  as  conducted  by  Klein  Associates  researchers  is  an  in-depth  examination  of  an  individuals  expertise 
in  the  context  of  his  or  her  job  the  cues  and  patterns  of  cues,  strategies,  challenges,  discriminations, 
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assessments,  expectancies,  goals,  and  ability  to  detect  and  anticipate  problems.  These  methods  are  in 
contrast  to  other  more  generic  or  abstract  CTA  methods  derived  from  a  task  analysis  approach.  Rather 
than  decompose  the  task  into  subcomponents,  the  intent  of  our  CTA  methods  is  to  get  inside  the  head  of 
the  expert  and  understand  the  rich  cognitive  elements  that  are  difficult  to  articulate.  For  The  CTA  tools 
we  used  to  uncover  these  issues  include  the  Critical  Decision  Method  (CDM)  and  the  Knowledge  Audit, 
and  are  described  below. 

3.2.1  Critical  Decision  Method 

Klein  Associates  most  frequently  used  research  tool,  the  Critical  Decision  Method  (CDM),  has  been  used 
in  dozens  of  studies  of  decision-making  and  problem  solving.  CDM  interviews  are  based  on  Flanagan's 
(1954)  critical  incident  technique  and  are  organized  around  an  initial,  unstructured  account  of  a  specific 
incident.  The  incident  account  is  generated  by  the  interviewee  in  response  to  a  specific  open-ended 
question  posed  by  the  interviewers,  and  it  provides  the  structure  for  the  interview  that  follows.  By 
requesting  accounts  of  a  certain  type  of  event,  and  structuring  the  interview  around  that  account, 
potential  interviewer  biases  are  minimized.  Once  the  report  of  the  incident  has  been  completed,  the  CDM 
interviewer  leads  the  participant  back  over  his  or  her  incident  account  several  times,  using  probes 
designed  to  focus  attention  on  particular  aspects  of  the  incident  and  solicit  information  about  them. 

Solicited  information  depends  on  the  purpose  of  the  study,  but  might  include  presence  or  absence  of 
salient  cues  and  the  nature  of  those  cues,  assessment  of  the  situation  and  the  basis  of  that  assessment, 
expectations  about  how  the  situation  might  evolve,  goals  considered,  challenges  faced,  and  options 
evaluated  and  chosen.  And  because  information  is  elicited  specific  to  a  particular  decision  and  incident, 
the  context  in  which  the  decision  maker  is  operating  in  remains  intact  and  becomes  part  of  the  data 
record. 

CDM  has  been  highly  successful  in  eliciting  perceptual  cues  and  details  of  judgment  and  decision 
strategies  that  are  generally  not  captured  with  traditional  reporting  methods,  and  has  been  demonstrated 
to  yield  information  richer  in  variety,  specificity,  and  quantity  than  is  typically  available  in  experts’ 
unstructured  verbal  reports  (Crandall,  1989).  The  information  obtained  via  these  methods  is  concrete  and 
specific,  reflects  the  point  of  view  of  the  decision  maker,  and  is  grounded  in  experience.  Detailed 
descriptions  of  CDM  and  other  work  surrounding  it  can  be  found  in  Klein  et  al.  (1989)  and  Hoffman  et 
al.  (1998). 

3.2.2  Knowledge  Audit 

Another  CTA  tool  is  the  Knowledge  Audit,  developed  under  a  contract  with  the  Navy  Personnel 
Research  and  Development  Center  (Militello  et  al.,  1997).  The  objective  was  to  develop  a  streamlined  set 
of  CTA  tools,  which  could  be  used  effectively  by  people  outside  of  the  cognitive  research  community. 
The  Knowledge  Audit  focuses  on  the  categories  of  knowledge  and  skills  that  distinguish  experts  from 
others.  These  categories  include  metacognition,  mental  models,  perceptual  cues  and  patterns,  analogues, 
and  declarative  knowledge.  The  Knowledge  Audit  provides  an  efficient  method  for  surveying  the  various 
aspects  of  expertise.  The  method  does  not  attempt  to  find  whether  each  component  of  expertise  is  present 
for  a  given  task.  Rather,  it  employs  a  set  of  specific  probes  designed  to  describe  the  type  of  knowledge  or 
skill  and  to  elicit  examples  of  each  based  on  actual  experiences.  The  primary  strength  of  the  Knowledge 
Audit  is  that  it  enables  us  to  survey  rapidly  the  nature  and  breadth  of  skills  involved  in  expertise 
in  a  given  domain. 
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The  next  section  will  describe  the  Artificial  Intelligence  (AI)  methods  that  were  used,  in  conjunction  with 
data  derived  from  the  CTA  methods,  to  build  an  advanced  decision-support  tool  for  the  Landing  Signal 
Officer  workstation.  CTA  data  was  a  critical  ingredient  in  building  an  intelligent  system,  and  a  more 
detailed  account  of  how  the  CTA  data  fit  into  AI  models  of  design  is  described  in  later  sections.  Next  is 
a  description  of  AI  methods  utilized  in  this  project. 

3.3  Case-Based  Reasoning 

Case-Based  Reasoning  (CBR)  reasoning  is  a  knowledge  representation  and  control  methodology  based 
upon  previous  experiences  and  patterns  of  previous  experiences.  These  previous  experiences  (previous 
carrier  landings),  or  “cases”  of  domain-specific  knowledge  and  action,  are  used  in  comparison  with  new 
situations  or  problems.  These  past  methods  of  solution  provide  expertise  for  use  in  new  situations  or 
problems. 

Much  of  the  research  in  Case-Based  Reasoning  is  directed  toward  retrieving  similar  cases  and 
determining  useful  definitions  of  similarity.  It  was  found  to  be  useful  in  retrieving  similar  or  relevant 
past  approaches.  For  a  pilot  trending  system,  the  cases  are  simply  previous  examples  of  carrier  landings, 
including  all  information  available  from  the  ship’s  systems,  from  which  inferences  and  comparisons  can 
be  made  using  CBR. 

Because  CBR  is  based  on  the  ways  humans  think,  it  is  a  very  natural  way  to  support  the  human  decision¬ 
making  process.  For  example,  the  LSOs  all  described  how  important  it  was  to  have  seen  several  previous 
approaches  made  by  the  incoming  pilot.  The  CBR  system  aids  this  process  by  retrieving  similar 
approaches,  in  case  this  particular  LSO  does  not  have  much  experience  with  this  particular  pilot,  for 
these  conditions. 

By  retrieving  a  set  of  veiy  similar,  very  relevant  cases,  the  CBR  system  helps  the  LSO  make  qualitative 
assessments  of  the  current  approach  based  on  the  particular  pilot’s  tendencies  and  trends.  This 
assessment  is  based  on  the  statistics  of  the  retrieved  similar  cases,  weighted  based  on  their  degree  of 
similarity  to  the  current  situation. 

3.4  Fuzzy  Logic 

Zadeh  introduced  the  concept  of  fuzzy  logic  in  1965.  Since  then,  fuzzy  logic  has  advanced  in  a  wide 
variety  of  disciplines  such  as  control  theory,  topology,  linguistics,  optimization,  and  category  theoiy. 
Unlike  a  crisp  set,  a  fuzzy  set  allows  partial  membership.  Fuzzy  logic  is  a  generalization  of  the 
traditional  TRUE/FALSE  bi-level  logic,  one  that  allows  for  non-sharp  transition,  representing  a  region  of 
partial  truth,  between  absolute  true  and  absolute  false.  For  example,  although  the  assertion  that  an 
individual  is  male  is  either  true  or  false  (and  is  therefore  crisp),  the  assertion  that  an  individual  is  lean  is 
not  so  clear-cut.  Figure  1  demonstrates  how  the  fuzzy  sets  may  be  used  to  capture  this  concept.  A  person 
with  a  body  fat  percentage  of  16.5  has  membership  values  of  0.12  and  0.43  in  the  "lean"  and  "moderately 
overweight"  fuzzy  sets,  respectively. 

The  basic  architecture  of  a  fuzzy  logic  data  analysis  system  is  illustrated  in  Figure  1.  The  numerical 
input  data  is  codified  through  the  fuzzifier  into  the  equivalent  linguistic  parameters  (such  as  lean, 
moderately  overweight,  and  obese),  with  associated  membership  function  values.  The  inference  engine 
uses  the  knowledge  in  a  particular  representation  to  derive  some  expert  conclusion  or  offer  expert  advice. 
It  includes  the  system’s  general  problem-solving  knowledge.  Various  rules  in  the  knowledge  base  and 
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decision-making  logic  are  invoked  and  recover  the  decision  actions  with  different  degrees  of  emphasis 
depending  on  their  respective  membership  values. 


Figure  1.  Fuzzy  Membership  Functions 


The  final  stage  in  the  fuzzy  logic  data  processor  aggregates  all  the  inferred  fuzzy  data  and  produces  an 
appropriate  conclusion  or  classification  of  the  system’s  input.  If  the  system’s  output  needs  to  be  in  non- 
fuzzy  numerical  format,  it  is  the  responsibility  of  the  defuzzification  module  to  convert  fuzzy  data  to 
numerical  from,  see  Figure  2. 


Figure  2.  General  architecture  of  fuzzy  logic  data  analysis  system 

Fuzzy  logic  was  found  to  be  useful  in  conjunction  with  case-based  reasoning  to  determine  similar 
landings.  In  addition,  fuzzy  logic  was  found  beneficial,  when  combined  in  a  neural  network  type  system, 
in  the  prediction  of  future  aircraft  locations. 

3.5  Neural  Networks 

Neural  networks  are  an  approach  to  machine  learning  which  developed  out  of  attempts  to  model  the 
processing  that  occurs  within  the  neurons  of  the  brain.  By  using  simple  processing  units  (neurons), 
organized  in  a  layered  and  highly  parallel  architecture  (see  Figure  3),  it  is  possible  to  perform  arbitrarily 
complex  calculations.  Learning  is  achieved  through  repeated  minor  modifications  to  selected  neurons, 
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which  results  in  a  very  powerful  classification  system.  Neural  network  software  is  used  to  recognize,  and 
also  to  run  at  desired  conditions.  Applications  include  handwriting  recognition,  fingerprint 
identification,  control  of  chemical  processes,  speech  recognition,  credit  analysis,  scientific  analysis  of 
data,  and  in  neurophysiological  research.  Neural  networks  are  also  referred  to  as  neural  nets, 
connectionism,  and  parallel  associative  memory. 


Neural  networks  techniques  were  utilized  in  conjunction  with  fuzzy  logic  techniques  to  create  a  neural 
network  based  fuzzy  inference  system  for  learning  predictions  for  future  aircraft/pilot  locations. 


ftpirf 

vector 


Figure  3.  An  Adaptive  Network 


3. 6  Neural  Network  Based  Fuzzy  Inference  System 

A  neural  network  based  fuzzy  inference  system  (Figure  4)  is  a  multi-layer  network  in  which  each  node 
performs  a  particular  function  (e.g.,  a  fuzzy  function)  on  incoming  signals  (as  well  as  a  set  of  parameters 
pertaining  to  the  node).  The  nature  of  the  node  function  may  vary  from  node  to  node,  and  the  choice  of 
each  node  function  depends  on  the  overall  input-output  function,  which  the  neural  network  is  required  to 
cany  out.  A  neural  network  has  two  types  of  nodes:  an  adaptive  node  (represented  by  a  square  in  Figure 
4)  has  parameters  that  may  be  updated  by  a  learning  algorithm,  while  a  fixed  node  (represented  by  a 
circle)  has  none.  A  neural  network-based  fuzzy  inference  system  is  comprised  of  several  layers  of  nodes, 
as  illustrated  in  Figure  4.  The  node  function  of  each  node  in  the  premise  layer  of  nodes  is  a  fuzzy 
membership  function,  which  specifies  the  degree  to  which  the  node’s  input  parameter  satisfies  some 
linguistic  quantifier  associated  with  the  node.  The  II  layer  of  nodes  outputs  the  firing  strength  of  the 
fuzzy  rules,  and  the  N  layer  normalizes  the  firing  strengths.  The  consequent  layer  performs  (Sugeno- 
type)  defuzzification,  aggregated  by  a  single  weighed  sum  node  in  the  final  layer.  Fuzzy  IF-THEN  rules 
that  the  system’s  structure  is  based  on  may  be  obtained  from  human  experts  or  constructed  automatically 
based  on  the  format  of  training  data.  The  learning  rule  is  a  hybrid  of  gradient  descent  and  least  square 
estimation  of  parameters.  In  the  forward  pass  of  the  learning  algorithm,  signals  go  forward  till  layer  4 
and  the  consequent  parameters  are  identified  by  the  least  squares  estimate.  In  the  backward  pass,  the 
error  rates  propagate  backward  and  the  premise  parameters  are  updated  by  gradient  descent. 

The  neural  network  based  fuzzy  inference  system  was  trained  using  various  landing  passes  and  learned  to 
predict  future  aircraft  locations. 
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:  ■■  consequent  parameters 

premise  parameters  *  ! 


Figure  4.  Neural  Network  based  Fuzzy  Inference  System 


4  Phase  II  Objectives  and  Accomplishments 

4.1  Objectives 

The  primary  objective  was  to  buttress  LSO  decision  making  by  developing  pilot  trending  and  ship 
oscillation  recognition  decision  aids.  In  support  of  this  primary  objective  are  several  subsidiary  ones: 

•  Elicit  Important  Pilot  Trending  and  Decision  Support  Considerations 

•  Elicit  Important  Approach  Parameters  and  Similarity  Measures 

•  Elicit  Important  Pilot  Considerations  and  Similarity  Measures 

•  Develop  Pilot  Trending  and  Ship  Oscillation  Recognition  Techniques/Software 

•  Design/Implement  LSO  Interface  for  Pilot  Trending  and  Ship  Oscillation 

•  Test  Prototypes 

4.2  Data  Collection 

During  the  early  stages  of  this  Phase  II  project  we  continued  Cognitive  Task  Analysis  (CTA)  of  the 
landing  signal  officers  (LSOs)  and  began  developing  an  LSO  interface.  The  envisioned  interface  design 
was  designed  to  provide  the  LSOs  with  critical  information  in  appropriate  formats  to  better  support  their 
understanding  of  how  the  pilots  trends  and  the  oscillations  of  the  ship  are  influencing  the  current 
recovery  for  a  particular  pilot  under  particular  circumstances  (i.e.,  help  them  land  the  planes  more  safely 
and  expediently,  a  somewhat  contradictory  LSO  mandate).  Our  approach  was  based  on  decision-centered 
design  concepts  wherein  the  critical  decisions  and  judgments  required  of  the  LSOs  drove  the  design 
development. 
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4.2.1  Knowledge  Elicitation 
Our  primary  subject  matter  experts  for  the  CTA  were  U.S  Navy  Commander,  Frank  Pfeiffer  (ret.),  a 
former  CAG  LSO,  and  the  instructors  at  the  LSO  Training  Center  at  NAS  Oceana,  Virginia  Beach,  VA. 
CDR  Pfeiffer  served  as  a  U.S.  Naval  Pilot  and  LSO  for  many  years  and  is  currently  employed  as  a  pilot 
for  a  major  commercial  airline.  During  the  Phase  I  and  Phase  n,  we  interviewed  CDR  Pfeiffer  on 
multiple  occasions  totaling  around  100  hours.  Many  of  the  sessions  were  one  to  two  days  in  length.  In 
addition  to  sessions  with  CDR  Pfeiffer,  we  have  visited  the  LSO  School  on  about  12  different  occasions 
where  we  performed  knowledge  elicitation  on  both  the  instructors  and  students,  and  had  them  react  and 
comment  on  design  concepts  and  previous  data  analyses  we  had  conducted.  In  the  early  stages  of  the 
project,  the  LSO  School  personnel  were  additional  sources  of  data  and  in  the  latter  stages,  they  served 
primarily  for  evaluation  (providing  feedback  about  the  display)  and  testing  of  the  interface  designs, 
which  will  be  described  later  in  the  document. 

We  employed  the  Critical  Decision  Method  (CDM)  and  Knowledge  Audit  to  elicit  critical  pilot  trending 
and  aircraft  and  ship  oscillation  data,  and  used  more  informal  interviewing  methods  to  elicit  background 
information  about  the  overall  LSO  task  environment.  For  pilot  trending  we  elicited  both  the  knowledge 
relating  to  an  approach  that  is  important  for  the  LSO  to  know,  and  how  to  identify  similar  approaches. 
Foremost,  we  elicited  important  approach  parameters  and  similarity  measures  (i.e.,  what  information 
about  the  approach  is  most  important  to  the  LSO,  and  how  is  similarity  between  approaches  defined). 
This  knowledge  was  needed  to  determine  what  should  be  displayed  to  the  LSO  and  when  it  should  be 
retrieved.  We  also  elicited  important  pilot  considerations  and  similarity  measures  (i.e.,  what  knowledge 
about  a  pilot  is  most  important  to  the  LSO,  and  how  is  similarity  between  pilots  defined,  for  purposes  of 
aiding  the  LSO). 

To  truly  employ  decision-centered  design,  we  needed  to  begin  by  understanding  how  LSOs  with 
experience  and  expertise  break  down  their  job  from  a  cognitive  perspective.  The  results  of  the  Phase  I 
CTA  revealed  that  there  were  several  key  types  of  information  that  were  critical  to  the  judgment/decision 
required  of  the  LSOs  to  perform  their  job  successfully,  and  we  used  these  as  a  springboard  for  further 
elicitation  in  the  Phase  II.  These  include: 

Deviations  of  the  aircraft  glide  path  from  the  glideslope,  dictated  by  the  basic  angle  of  the  day. 

Deviations  of  the  aircraft  speed  from  the  ideal,  based  on  its  attitude  and  the  particular  aircraft  type. 
Deviations  of  the  aircraft  line  up  from  the  centerline  of  the  landing  area  on  die  angled  deck. 

Abnormal  aircraft  configuration,  based  on  the  aircraft  type. 

It  is  important  for  us  to  stress  that  the  above  list  identifies  critical  information  requirements  of  LSOs,  but 
these  should  not  be  confused  with  decision/judgment  requirements.  The  decisions  and  judgments  will 
provide  us  contexts  within  which  these  data  are  appropriate  and  will  provide  frames  for  design  concepts. 
This  will  be  addressed  more,  later  in  the  report. 

For  each  of  the  above  points  (glide  path,  attitude,  line  up,  and  configuration)  the  CTA  in  the  Phase  II  also 
brought  out  the  important  conditions  under  which  the  landings  were  occurring  (Case  I,  II,  or  ID). 

•  Case  I  -  day  with  good  weather,  visibility,  and/or  sea  conditions. 

•  Case  II  -  night  with  good  weather,  visibility,  and/or  sea  conditions. 

•  Case  HI  -  poor  weather,  visibility,  and/or  sea  conditions. 
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In  addition,  two  other  criteria  were  identified  as  important  influences  on  the  decision  making  of  the  LSO: 
the  type/model  of  aircraft  and  the  experience/competence  of  the  pilot.  The  type/model  of  aircraft  dictates 
the  characteristics  and  responsiveness  of  the  aircraft.  This  in  turn  influences  how  much  the  LSO  can  let  it 
drift  off  centerline  (the  larger  the  wingspan,  the  less  drift  that  can  be  allowed),  how  quickly  the  engine 
can  spool  up  (the  slower  the  spool  up  time,  the  further  out  the  power  call  must  occur),  etc. 

Our  knowledge  elicitation  in  Phase  II  uncovered  that  the  LSOs  have  five  categories  they  use  to  classify 
the  experience/competence  of  the  pilots:  New  Guys  (FNG),  Average  Pilots,  Top  Pilots,  Problem 
Children  (those  pilots  who  were  having  a  lot  of  difficulties),  and  Staff/COD  pilots  (those  pilots  who  did 
not  get  to  land  that  often  on  the  carrier).  During  one  of  the  data  review  sessions,  the  LSOs  said  that  they 
treat  Problem  Children  and  Staff/COD  pilots  the  same  as  FNGs,  so  we  collapsed  these  three  into  the 
FNG  category,  leaving  3  categories:  FNG,  Average,  and  Top  Pilots. 

Furthermore,  the  Phase  I  data  collection  uncovered  that  the  LSOs  consistently  visualize  the  overall 
recovery  of  aircraft  by  segments  that  are  related  to  where  the  aircraft  is  in  its  approach  pattern.  In  the 
Phase  II,  the  LSOs  helped  us  determine  that  the  critical  segments  for  the  pilot  trending  components  are 
primarily  from  when  the  aircraft  is  roughly  one  mile  out  all  the  way  on  in  to  the  deck.  While  1  mile  in  is 
the  critical  area,  we  collected  data  for  the  entire  Case  I  &  II  approach  from  where  the  aircraft  first  makes 
the  pass  along  the  starboard  side  of  the  ship.  This  would  be  the  equivalent  of  greater  than  three  miles  out 
for  a  Case  EH  recovery.  The  LSO  terms  for  these  segments  are: 

•  The  Pass 

•  The  Break 

•  The  1 80  (Case  I  &  II  approaches)  or  3  NM  (Nautical  Miles)  out  (Case  IH). 

•  The  135  (Case  I  &  II  approaches)  or  2  NM  out  (Case  IH). 

•  The  90  (Case  I  &  II  approaches)  or  1  NM  (Case  HI). 

•  The  Start  (X).  Approximately  Va  NM  from  the  ramp. 

•  In  the  Middle  (IM).  Approximately  14  NM  from  the  ramp. 

•  In  Close  (IC).  Approximately  1/8  NM  from  the  ramp. 

•  At  the  Ramp  (AR).  Right  at  the  ramp  (stem  of  the  ship) 

•  In/Over  the  Wires  (IW  or  OW):  The  area  where  the  arresting  wires  cross  the  landing  area. 

IW  implies  the  aircraft  has  been  trapped  while  OW  implies  it  missed  the  wires. 

Note  that  from  the  Start  (X)  on  in,  the  three  Cases  are  identical  with  the  exception  of  the  lighting 
(day/night)  and  weather  conditions  (clear,  heavy  rain,  stormy,  etc.). 

Knowledge  elicitation  only  represents  one  aspect  of  the  CTA  methodology.  Further  analysis  and  data 
representation  are  necessary  in  order  to  make  sense  of  the  information  before  the  data  is  shared,  in  this 
case,  with  the  Artificial  Intelligence  experts,  and  subsequently  fed  into  AI  models  for  development  of  a 
decision  support  tool.  The  next  section  describes  how  the  data  was  organized  into  a  useful  form  that 
could  be  used  by  AI  technicians. 

4.2.2  CTA  Representation 

Representation  of  the  data  collected  from  the  interviews  is  a  crucial  aspect  of  the  CTA  methodology.  A 
Decision  Requirements  Table  (DRT)  was  created  to  organize  the  information  that  was  elicited  in  the 
CTA  interviews  (see  Appendix  A).  DRTs  are  a  popular  format  for  organizing  data,  especially  in  terms  of 
decision  requirements  and/or  the  information  needed  to  make  the  critical  decisions.  For  example,  we 
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collected  data  about  how  an  aircraft  can  deviate  from  the  glide  slope  (from  dimension  3),  under  Case  II 
conditions  (dimension  2)  within  flight  segment  IM  (dimension  1). 


We  refer  to  the  tabulation  of  information  across  dimensions  as  a  decision  requirements  table.  Table  1 
shows  a  sample  Decision  Requirements  Table  that  is  not  completed.  A  completed  decision  requirements 
table  for  the  LSO  is  found  in  Appendix  A.  The  decision  requirement  tables  and  cognitive  task  analyses 
from  the  Phase  I  and  Phase  II  identify  the  critical  decisions  and  judgments  around  which  all  design 
recommendations  are  focused  (i.e.,  decision-centered  design). 

The  following  probes  organize  the  DRT  table  illustrated  below: 

How  can  they  deviate  (e.g.,  go  high,  low,  left,  right,  etc.)? 

What  are  the  indicators  of  these  deviations  (angle  of  bank,  altitude,  etc.)? 

What  are  the  specific  cues/indicators  to  the  LSO  (how  much  right  wing  tip  is  visible,  etc.)? 

At  which  point  can  the  LSOs  discriminate  the  deviation  (e.g.,  30  ft.  off  glide  slope)? 

At  what  value  or  point  does  a  deviation  become  a  problem  (e.g.,  5  ft.  low)? 

Are  there  differences  (when  it  becomes  a  problem)  for  the  three  different  pilot  types? 

And  finally,  why  are  these  data  important  for  the  LSO?  That  is,  what  are  they  trying  to  do  that 
requires  them  to  need  this  information? 

Case  How  Can  Deviate 
Indicators 

Specific  Cues/Indicators 
Discrimination  Ability 

When  does  Deviation  Become  a  problem?  Differences  for  Pilots/Pilot  Types? 

Glide  Slope/Glide  Path 
1200  ft.  decent  begins  Attitude/ 

SpeedLine  UpConfiguration  of  aircraft 


Case 

How  Can 
Deviate 

Indicators 

Specific  Cues 
and 

Indicators 

Discrimination 

Ability 

When  does 
Deviation 
“Become  a 
problem?” 

Glide 

Slope/Glide 

Path 

1200  ft.  decent 
begins 

Attitude/ 

Speed 

Line  Up 
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Configuration 
of  aircraft 

Table  1:  Decision  requirement  table  example. 


4.2.3  Data  Sources 

In  parallel  to  the  CTA  work,  we  examined  several  other  data  sets.  These  data  sources  also  fed  into  the  AI 
models.  These  are  briefly  described  below: 

SPN-46  Radar  Data.  The  SPN-46  radar  tracks  several  aspects  of  each  approaching  aircraft  and  of  the  ship 
at  a  rate  of  approximately  15-20  samplings  per  second.  For  each  aircraft  it  collects  plane  position 
information  using  three  coordinates:  distance  from  the  ship,  horizontal  position  with  respect  to  the  ship 
(line  up),  and  altitude.  In  addition,  it  processes  some  of  these  data  to  also  provide  the  aircraft’s  closing 
speed  and  its  sink  speed.  The  SPN-46  radar  also  tracks  the  ships  pitch  and  roll  in  degrees.  Header  data 
includes  the  time-hack,  pass  number,  the  radio  channel  in  use,  and  aircraft  side  number. 

LSO  Comments/Grades.  Every  pass  is  graded  by  the  controlling/backup  (senior)  LSO.  These  grades,  or 
comments,  describe  the  aircraft’s  location  and  characteristics  for  each  segment  of  the  approach,  at  least 
from  the  start  (X)  in  to  the  wires.  An  annotated  example  of  a  LSOs  grade/comment  on  a  pass  is  provided 
in  Appendix  E.  A  glossary  of  the  LSOs  grading  is  included  in  Appendix  D. 

Automated  Performance  and  Readiness  Training  System  fAP ARTS!  Program.  Data.  &  Reports. 

APARTS  is  a  program  used  by  the  LSOs  to  input  records  of  every  recovery  pass.  It  works  from  a  MS 
ACCESS  database  and  can  generate  a  variety  of  reports,  both  for  the  individual  pilot  and  the  squadrons. 
A  key  piece  of  data  that  APARTS  uses  is  the  LSOs  Grade/Comments  for  each  pass.  It  is  from  these 
inputted  APARTS  data  that  our  system  will  extract  the  information  about  a  pilot’s  trend  and  history. 

Accident  Summary  Reports.  The  U.S.  Navy  Safety  Center,  located  at  the  Norfolk  Ship  Yards,  Norfolk, 
VA  keeps  a  complete  set  of  summaries  of  all  incidents  that  occur  in  the  U.S.  Navy.  We  requested  a  set  of 
recovery  incidents  involving  fixed  wing  aircraft,  where  the  LSO  was  mentioned.  Several  hundred 
incidents  were  retrieved.  We  studied  these  for  any  possible  patterns  and  information  that  might  prove  to 
be  useful  in  this  project.  An  example  of  a  report  is  presented  below.  Note  that  the  summary  report  is 
informative,  but  also  rather  cryptic.  They  did,  however,  give  us  a  feel  for  the  range  of  incidents  that 
occurs  on  the  carriers,  beyond  the  typical  ramp  strikes  we  had  learned  about  through  other  data  sources. 

Example  of  Naval  Safety  Center  Accident  Summary  Report 

ACFTMOD  EVENTSERL  FOMA  15753 

. EVENTSUMMARY . 

During  night  CV  bolter,  ACFT  drifted  right  &  stbd  wing  tip  impacted  2  ACFT  spotted  on  stbd 
side  of  foul  line.  ACFT  subsequently  recovered  safely.  Mishap  cause  factors:  aircrew  error 
improper  landing  technique.  Pilot  failed  to  correct  for  right  drift  at  the  ramp.  Line  up  corrections 
throughout  the  approach  were  timely  &  appropriate.  Just  after  crossing  the  ramp  ACFT 
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established  a  right  drift.  At  that  point  ACFT  was  waved  off  for  being  too  high  to  land  safely.  The 
pilot  transitioned  to  inside  the  cockpit  to  set  the  attitude  &  monitor  his  instruments.  Mp  stated  he 
sensed  the  right  drift  but  did  not  believe  it  to  be  excessive  &  initiated  no  correction  prior  to 
touchdown.  Controlling  personnel  factors  backup  LSO  failed  to  make  a  timely  line  up  call  when 
right  drift  was  noted  as  ACFT  crossed  the  ramp.  The  backup  LSO  is  responsible  for  monitoring 
ACFT  line  up  during  the  approach.  He  must  monitor  line  up  all  the  way  to  touchdown  &  be 
ready  to  give  a  mandatory  UHF  call  when  a  deviation  occurs,  even  if  the  ACFT  is  over  the  wires. 
The  backup  LSO  did  see  the  right  drift  at  the  ramp  but  elected  not  to  issue  a  line  up  call  fearing 
ACFT  would  touchdown  on  left  main  mount  only.  It  is  the  backup  LSO's  responsibility  to  call  for 
the  correction,  &  trust  the  pilot  to  be  aware  of  his  proximity  to  the  deck  &  make  the  correction 
appropriate  to  the  situation. 

Video  Tapes  of  Recovery  Incidents/Accidents.  The  instructors  at  the  U.S.  Navy  LSO  School  at  NAS 
Oceana,  Virginia  Beach,  VA  provided  us  with  a  video  compendium  of  carrier  recovery  incidents.  These 
were  extremely  informative,  and  sobering.  The  video  provided  us  with  a  sense  of  the  acute  time  pressure 
the  LSOs  work  within  and  the  dangerous  nature  of  the  aircraft  recovery  process. 

4.3  Pilot  Trending  Analysis 

To  perform  the  pilot  trending  research  it  was  required  that  we  get,  from  the  Navy,  data  associated  with  a 
large  number  of  carrier  landings,  for  the  full-range  of  aircraft  that  the  LSO  must  help  land.  We  tapped 
specific  data  sources  for  this  task,  which  included  SPN-46  data  (range,  bearing,  and  altitude  data  over 
time  for  the  incoming  aircraft),  and  ship  motion  data  (pitch,  roll,  heave,  and,  preferably,  velocity) 
retrieved  from  at-sea  landings.  The  most  critical  data  source  used  for  the  pilot  trending,  however,  was 
the  LSO  comments  and  grades,  which  already  existed  in  an  APARTS  data  base  for  each  pilot  (Appendix 
E). 


4.3.1  Case-Based  Reasoning  (CBR) 

We  applied  the  technique  of  Case-Based  Reasoning  (CBR)  to  address  the  Pilot  Trending  problem.  Much 
of  the  research  in  Case-Based  Reasoning  is  directed  toward  retrieving  similar  cases  and  determining 
useful  definitions  of  similarity.  For  a  pilot  trending  system,  the  cases  are  simply  previous  examples  of 
carrier  landings,  including  all  information  available  from  the  ship’s  systems,  from  which  inferences  and 
comparisons  can  be  made  using  CBR.  In  order  to  define  what  constitutes  similarity  between  approaches 
and  between  pilots  from  the  LSO  perspective,  the  LSO’s  notation  and  comments  provided  one  aspect  to 
this  representation.  These  comments  captured  an  approach’s  motion  pattern  at  a  high-level  of 
abstraction.  These  high  level  comments  then  were  used  as  a  basis  for  assessing  the  similarity  between 
two  approaches. 

From  another  perspective,  case  representations  often  include  features  at  a  low  level,  and  features  at  a 
high  level  of  abstraction.  For  approaches,  the  low  level  features  included  the  approach  data  (range, 
bearing,  altitude),  and  the  high-level  features  included  the  LSO’s  comments.  Obviously,  since  many  of 
the  LSO’s  comments  capture  the  approach  at  a  high-level  of  abstraction,  there  was  some  redundancy  of 
information. 

CBR  uses  cases  to  record  the  experience,  know-how  and  process  the  reasoning  for  retrieving  solutions 
from  such  subsequently.  A  case  is  a  contextualized  piece  of  knowledge  representing  an  experience.  It 
contains  a  past  lesson  that  is  the  content  of  the  case  and  the  context  in  which  the  lesson  can  be  used.  In 
our  project  a  case  consists  of  the  LSO  comments  of  the  flight  performance  at  different  stages  as  the 
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specific  aircraft  approaches  the  aircraft  carrier  under  the  same  pilot  and  the  same  environmental 
conditions. 

The  CBR  software  delivers  two  major  objectives  and  show  its  results  on  the  display: 

•  Approaches  based  on  similar  conditions:  Retrieval  of  the  number  of  approaches  with  similar 
conditions  and  the  total  number  of  traps  per  pilot  with  this  aircraft  type. 

•  Trend  Patterns:  Retrieval  of  similar  landing  trend  patterns  from  stored  cases;  display  them 
graphically;  and  show  the  LSO  comments  of  the  closest  case. 

The  case-based  reasoning  (CBR)  for  pilot  trending  consists  of 

1)  Indexing, 

2)  Similarity  definition,  and 

3)  Retrieval  algorithm. 

We  use  CBR  to  represent  the  pilot  trending  knowledge  and  use  that  knowledge  for  LSOs  to  evaluate  the 
landing  performance;  and  to  provide  related  trending  flight  cases.  Through  CBR,  we  represent  prior 
approaches  and  the  affiliated  LSO  comments  as  cases.  Retrieval  of  similar  cases  is  then  performed  to 
provide  the  similar  cases  for  pilot  trending  analysis.  Upon  reading  the  pilot  information,  the  current 
weather  condition,  and  the  aircraft  position  (SPN-46),  the  pilot  trending  system  uses  the  case  base 
reasoning  system  to  retrieve  the  most  similar  patterns  from  previous  cases  stored  in  the  APARTS  historic 
database.  The  recent  similar  10  patterns,  and  the  current  one  are  displayed  on  the  displayed  panel  in 
graphical  format.  The  associated  LSO  comments  may  also  be  displayed. 

The  overall  pilot  trending  architecture  is  as  depicted  in  Figure  5.  The  Pilot  Trending  Analysis  box  in 
Figure  5  is  handled  via  case-based  reasoning. 


Figure  5.  The  Architecture  of  Pilot  Trending  System 
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4.3.1.1  Indexing 

Several  features  are  selected  in  defining  the  indexes  for  the  cases.  Indexing  facilitates  retrieval  of  similar 
cases.  The  multiple  features  selected  as  indexes  are  pilot  name,  aircraft  type,  glideslope,  lineup,  and 
day/night. 

4.3.1. 2  Similarity  Definition 

Similarity  assessment  is  the  process  of  comparing  an  incoming  flight  pattern  to  stored  approach  cases 
using  the  similarity  definition  and  indexes  to  produce  a  similarity  score.  This  is  done  progressively  as  the 
aircraft  approaches  the  aircraft  carrier.  As  more  flight  pattern  data  is  available  during  the  pass  as  the 
aircraft  progresses  from;  at  the  start  (X),  to  in  the  middle  (IM),  to  in  close  (IC),  and  to  at  the  ramp  (AR), 
the  similarity  assessment  is  performed  for  each  of  the  fours  stages. 


Figure  6.  Similarity  assessment  with  glideslope  matching 


As  shown  in  Figure  6  the  pattern  matching  degree  is  evaluated  by  comparing  the  current  glideslope 
pattern  with  that  of  the  stored  ones.  A  similar  procedure  is  applied  to  lineup  before  the  system  decides 
what  stored  case  to  be  retrieved  as  the  closest  one. 


The  above  similarity  definition  takes  the  following  into  consideration. 

i)  If  the  glideslope  difference  between  the  current  one  and  that  of  the  stored  one  is  small,  it  will 

have  a  relatively  small  matching  index  value  and  is  therefore  considered  close. 
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ii)  At  each  stage  the  matching  index  takes  a  weighted  sum  on  the  mismatch  of  the  current 

stage  and  that  of  the  accumulate  mismatch  of  the  previous  stages. 

The  matching  index  is  a  direct  measure  on  how  close  the  stored  case  is  to  the  incoming  flight.  With  the 
proper  normalization  of  the  linguistic  to  numeric  conversion,  e.g., 

•  Very  High  =  0.5 

•  Medium  High  =  0.33 

•  Little  High  =  0.17 

•  OK  =  0 

•  Little  Low  =  -0.17 

•  Medium  Low  =  -0.33 

•  Very  Low  =  -0.5. 

Flight  Path  Deviations  to  Linguistic  Conversion 

For  flight  data  that  does  not  have  associated  linguistic  data,  PADAL  has  to  determine  the  appropriate 
linguistic  conversion  from  numerical  flight  path  data.  Fuzzy  logic  is  employed  in  PADAL  to  perform 
flight  path  to  linguistic  conversion.  Fuzzy  lineup  and  glideslope  functions  are  represented  in  Figure  7. 
The  lineup  category  consists  of  7  fuzzy  sets,  ranging  from  significant  left  lineup  (_LUL_)  to  significant 
right  lineup  (_LUR_).  The  glideslope  category  is  subdivided  into  7  analogous  fuzzy  sets,  which 
construct  a  “very  high”  (_H_)  to  “very  low”  (_LO_)  classification  of  the  aircraft’s  glideslope.  These 
fuzzy  sets  map  directly  onto  the  comments  used  by  LSOs  to  describe  the  aircraft’s  position. 


Figure  7.  Lineup  &  Glideslope  Fuzzy  Membership  Functions 


Similar  fuzzy  definitions  are  constructed  for  various  other  parameters  that  define  the  landing  trajectory. 
These  fuzzy  concepts  enable  the  system  to  classify  any  point  in  the  landing  trajectory  by  associating 
fuzzy  membership  values  with  it. 


24 


For  example,  a  marked  point  in  Figure  8  has  the  glideslope  deviation  from  the  nominal  glideslope  (3.5°) 
of  3.7314°-3.5°  =  0.2314°,  which  corresponds  to  the  following  glideslope  classification: 


Glideslope: 

P_lo_  =  0.00 

=  0.39 

ftLO 

=  0.00 

ftH 

=  0.93 

M(lo) 

=  0.00 

=  0.27 

^PERFECT-  0.00 

This  means  that  an  aircraft  in  that  position  is  very  likely  to  be  classified  as  high  by  a  landing  signal 
officer,  somewhat  likely  to  be  classified  as  very  high  or  a  little  high,  and  extremely  unlikely  to  be 
classified  as  low. 

4.3. 1.3  Retrieval  Algorithm 

After  the  case-based  reasoner  has  determined  the  most  similar  landings,  the  retrieval  algorithm  retrieves 
these  similar  landings  from  stored  cases;  displays  them  graphically;  and  displays  the  LSO  comments  of 
the  closest  case.  Based  on  the  information  of  the  incoming  flight,  the  CBR  system  performs  and  provides 
the  following  for  display  on  the  display  panel. 

•  Total  traps  are  the  number  of  cases  that  have  the  pilot  name  index  matching  that  of  the  incoming 
pilot. 

•  Similar  conditions  are  the  number  of  cases  that  have  the  index  (pilot  name,  aircraft  type, 
squadron,  and  day/night)  matching  that  of  the  incoming  flight. 
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In  addition,  the  ten  most  recent  and  similar  stored  cases  that  match  the  current  flight  pattern  will  be 
displayed  in  four  panes,  corresponding  to  X,  IM,  IC,  and  AR  stages  respectively,  with  the  following 
features. 

i)  A  graphical  representation  of  the  past  ten  similar  flights’  trend  will  be  displayed 

ii)  Four  graphical  windows  will  be  aligned  with  the  LSO  comment  summary  for  each  of  the 
four  stages,  i.e.,  X,  IM,  IC,  and  AR. 

iii)  Each  window  contains  the  1 0  most  similar  passes. 

iv)  In  each  window,  there  will  be  a  Cartesian  coordinate  with  the  horizontal  axis  showing 
the  lineup  and  the  vertical  axis  representing  the  glideslope. 

v)  The  trend  data  will  be  represented  as  dots  of  vaiying  sizes  (similarity).  Larger  dots  show 
the  most  similar  ones. 

4.4  Prediction  of  Plane  Trajectory  &  Ship  Motion 

To  guide  an  aircraft  to  land  more  safely  and  smoothly  aboard  aircraft  carriers.  Landing  Signal  Officers 
(LSO)  on  board  need  to  advise  incoming  pilots  to  adjust  their  flight  patterns  continuously.  The  ability  to 
predict  how  the  aircraft  motion  trajectory  may  look  can  facilitate  LSOs  in  making  their  guiding  decision. 
Typically,  the  flight  pattern  is  carefully  observed  and  guided  when  aircraft  is  within  one  nautical  mile  (1 
NM)  from  the  landing  deck  in  open  sea.  This  corresponds  to  slightly  more  than  one  minute  in  real  flight 
time.  A  radar  system  records  all  the  trajectories  of  different  pilots  flying  various  aircraft.  This  data  may 
be  used  to  train  a  system  for  subsequent  prediction  purposes.  A  projection  2  seconds  ahead  of  the  current 
flight  position  is  usually  considered  appropriate.  Another  useful  subject  that  helps  LSOs  in  this  guiding 
process  is  the  prediction  of  the  ship’s  deck  motion  in  the  forthcoming  4  seconds.  If  the  deck  is  predicted 
to  tilt  up,  an  LSO  can  advise  the  pilot  to  land  somewhat  higher  as  it  touches  the  deck.  Misleading 
prediction  may  lead  to  a  crash  or  waveoff.  A  reliable  prediction  algorithm  is  therefore  essential  for  this 
task. 

This  task  consisted  of  solving  a  time  series  prediction  problem  in  which  past  and  present  motion  profiles 
are  provided  to  the  prediction  system  to  predict  the  motion  in  the  next  few  seconds.  No  other 
information  was  provided  to  base  the  prediction  on,  such  as  present  engine  setting  or  wind  speed  and 
direction.  Thus  die  general  problem  was  to  take  as  input  noisy  time-series  profiles  with  a  maximum 
duration  of  about  1  minute  and  provide  a  2  second  hence  prediction  of  the  plane’s  location.  This 
problem  may  be  depicted  as  in  Figure  9. 


Figure  9.  Time  Series  Prediction 


The  data  set  in  the  aircraft-landing  domain  consists  of  numeric  aircraft  trajectory  curves  recorded  by  a 
radar  positioned  aboard  a  ship.  The  radar  monitors  landing  aircraft’s  lineup  (its  horizontal  distance  from 
the  center  of  the  deck)  and  glideslope  (its  approach  angle).  The  data  provided  contains  substantially  noise 
(see  Figure  10),  and  the  magnitude  of  the  noise  varied  amongst  the  individual  passes.  Since  the  data 
includes  noise  and  we  do  not  know  nature  of  the  noise,  the  difficulty  of  the  problem  is  significantly 
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increased.  Potential  candidates  for  solution  of  this  problem  included  statistical,  physics-based,  Fourier, 
wavelet,  neural  networks,  fuzzy  logic  and  other  transform  and  machine  learning  techniques. 


101-394 


Figure  10.  Lineup  Position  Data  with  Significant  Noise 

Many  of  the  just  mentioned  candidates  were  investigated  to  varying  degrees  during  both  the  initial  Phase 
II  and  also  during  the  Option  portion  of  the  Phase  D.  The  most  successful  of  the  techniques  are  described 
below  and  they  are  incorporated  into  the  PADAL  software.  The  first  technique  described  is  referred  to 
as  ANFIS  in  the  software  and  uses  a  combination  neural  networks  and  fuzzy  inference.  The  second 
technique  described  is  referred  to  as  Velocity  in  the  software  and  is  a  physics-based  technique  developed 
during  the  Option  portion  of  the  Phase  n. 

4.4.1  ANFIS  (adaptive-network-based  fuzzy  inference  system) 

ANFIS  is  a  machine  learning  based  system  that  is  trained  with  past  data  before  it  is  engaged  in  the 
prediction  task.  The  system  is  trained  using  noisy  landing  profiles,  each  lasting  approximately  1  minute. 
After  training  the  system  with  past  profiles,  the  system  is  exposed  to  time  series  input  and  forecast  occur 
for  the  position  2  seconds  into  the  fixture.  All  the  landing  trajectory  curves  considered  in  the  course  of 
present  study  are  subdivided  into  five  categories  based  on  the  landing  aircraft  type:  F-14A,  F-14B,  F-18, 
A-6,  and  C-2.  These  categories  provide  a  natural  way  of  subdividing  the  original  trajectories  into 
modules.  In  addition  to  the  radar-recorded  data,  new  automatically  generated  curves  (grouped  into 
modules)  were  added  to  the  system  in  order  to  determine  how  the  size  of  the  data  set  affects  the 
performance  of  the  modular  design.  These  curves  were  produced  by  a  linear  convolution  of  the  original 
curves  within  each  aircraft  category. 

Typically,  the  input  motion  profiles  can  be  clustered  into  several  loosely  coupled  categories.  This  can  be 
the  basis  of  modular  decomposition.  The  landing  motion  profiles  of  F14A,  F14B,  FI  8,  A6,  and  C-2,  for 
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example,  are  different  though  they  share  some  common  characteristics.  Figure  1 1  depicts  the  input  space 
of  five  such  interconnected  clusters. 


Figure  11.  Clustered  Data  in  Input  Space 

This  modular  nature  of  the  input  data  space  was  used  to  modularize  the  design  in  the  learning  stage  by 
training  different  neural  network  based  fuzzy  inference  systems  with  respect  to  each  input  data  category. 
One  can  take  further  advantage  of  the  parallel  processing  technique  to  reduce  the  computation  time. 
Aggregation  of  these  individually  trained  modules  produced  one  generalized  module  for  prediction 
purpose.  This  generalized  module  is  expected  to  have  prediction  performance  comparable  to  that  of  the 
system  trained  with  the  traditional  non-modular  approach.  The  computation  effort  and  the  design 
complexity  are  both  expected  to  be  drastically  lower  with  the  proposed  modular  approach. 

The  neural  network  based  fuzzy  inference  system’s  ability  to  construct  an  input-output  mapping  in  a  fast 
and  efficient  manner  was  one  of  the  many  factors  that  led  to  its  selection  for  the  aircraft  prediction 
problem.  The  system  was  trained  with  a  subset  of  the  past  data  before  it  was  engaged  in  the  on-line 
prediction  task.  After  training  the  system  with  a  subset  of  the  past  profiles,  the  system  was  exposed  to 
unforeseen  approaches  and  forecast  its  profile  in  the  next  few  seconds  on-line. 

Figure  12  shows  a  sample  aircraft  lineup  trajectory  (filtered  position),  the  trajectory  predicted  by  the 
adaptive-network-based  fuzzy  inference  system  (ANFIS),  and  the  trajectory  predicted  by  a  1st  order 
polynomial  extrapolation  based  upon  the  most  recent  several  seconds  of  the  trajectory  (poly  1).  The  y 
value  of  each  of  the  two  prediction  curves  at  time  t  shows  the  position  that  was  predicted  2  seconds  into 
the  future  at  time  t-2.  As  is  typical  with  time  series  prediction  algorithms,  there  is  a  tradeoff  between 
algorithms  that  respond  quickly  to  changes  in  recent  data  values  and  algorithms  that  are  tolerant  of  noise. 

We  tried  a  number  of  polynomial  prediction  algorithms  based  on  various  weightings  and  time  windows 
for  the  0th,  1st,  and  2nd  derivatives  of  the  most  recent  n  seconds  of  the  trajectory.  For  each  prediction 
algorithm,  we  used  graphical  analysis  of  the  predicted  trajectories  to  understand  the  types  of  prediction 
errors  characteristic  of  each  algorithm  (undershoot,  overshoot,  lag),  and  calculated  total  prediction  error 
across  the  duration  of  each  trajectoiy.  We  empirically  determined  that  the  polynomial  prediction  that 
exhibited  the  lowest  error  was  a  weighted  average  of  the  current  position  and  a  linear  (1st  order) 
extrapolation  of  the  last  several  seconds  of  the  trajectory.  That  is,  predicting  the  trajectory  using  2nd 
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order  or  higher  polynomial  terms  tended  to  degrade  the  prediction.  The  neural  network  based  fuzzy 
inference  system  outperformed  this  "best"  polynomial  prediction  algorithm. 

trajectory  ID 


filtered  position 
prediction  ANFIS 
prediction  poly  1 


Figure  12.  Comparison  of  neural  network  prediction  with  polynomial  prediction  for  4  sample 
trajectories 

4.4.2  Physics-Based  ‘Velocity’  Predictor 

The  ANFIS  predictor  developed  during  the  Phase  II  showed  good  performance,  however,  it  was  hoped 
that  even  better  performance  could  be  realized.  To  this  end  one  of  the  goals  of  the  Phase  II  Option 
became  the  further  investigation  of  aircraft  prediction.  Unfortunately,  no  further  landing  information 
was  provided,  that  is,  only  past  trajectory  information  is  available,  no  wind  speed,  no  plane  setting 
information  etc. 
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Again  a  survey  of  various  promising  techniques  were  explored.  The  end  result  is  a  traditional  physics- 
based  approach  looking  at  the  position,  velocity  and  acceleration  of  the  plane. 


4.4.2.1  The  Least  Square  Estimation 

With  no  external  disturbances  included  and  considering  the  plane  as  a  point  mass,  the  plane’s  lineup 
motion  will  be  assumed  in  the  form  of: 

A  A  .  A  A  2 

y  =  y0  +  vyt  +  ayt 

Starting  with  the  specific  case  of  zero  acceleration  results  in: 

A  A  A 

y  =  y0  +  vyt 


This  is  the  simple  form  of  a  first  order  linear  equation.  The  least  square  estimation  problem  for  this 
equation  is  defined  as: 

lse  =£o -yJ2  =  T,(y»  “CP.  +  v/))2 

n=0  n~0 


The  goal  is  to  estimate  y0  and  vy,  which  minimizes  the  LSE.  A  variable  in  the  above  equation  is  N,  the 
number  of  samples  of  past  location  and  velocity  information;  the  determination  of  the  optimum  number 
of  samples  (N)  that  gives  the  best  estimation  of  %  and  vy  turns  out  to  be  the  greatest  challenge  in  using 
the  LSE.  Once,  the  values  of  y0  and  vy  are  determined,  the  2  second  hence  prediction  becomes  simply: 

yP  =;Po+v(f+2) 

Experiments  showed  that  with  ‘smaller’  window  sizes  (N)  the  prediction  tends  to  fluctuate  more  due  to 
the  noise  in  the  data,  resulting  in  increasing  LSE  error  for  the  smallest  values  of  N.  Conversely,  for 
‘larger’  window  sizes  the  prediction  gets  smother,  but  the  estimation  worsens  and  the  LSE  error  increases 
if  the  window  size  is  increased  too  much.  Therefore,  the  optimization  problem  becomes  one  finding  the 
value  of  N  that  minimizes  the  LSE  across  various  landings. 

Next  introduce  acceleration  term  back  into  the  equation. 

A  A  A  .  A  7 

y  =  y0  +  vyt  +  ayt 

It  was  hoped  that  the  acceleration  term  would  eliminate  some  of  the  lag  found  in  the  prediction  in  cases 
the  aircraft  changed  direction  on  the  order  magnitude  of  180  degrees.  Unfortunately,  the  prediction 
tended  to  degrade  because  the  noise  in  the  data  did  not  allow  for  determination  of  reliable  acceleration 
values. 

So  after  extensive  experimentation  it  was  concluded  that  the  best  overall  prediction  results  were  obtained 
by  the  simpler  equation  that  only  used  the  plane’s  position  and  velocity. 
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4.4.2. 2  Velocity  Estimation 
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We  estimated  the  velocity  based  on  available  position  data,  the  lineup  and  glideslope  data  were 
considered  separately.  The  equation  assumes  that  in  a  small  interval  the  velocity  in  the  lineup  direction 
does  not  change.  This  interval  is  the  interval  determined  by  minimizing  the  LSE  above. 

Vy„  =(yn-yn-k)/(tn  -tn-k) 


So  the  time  interval  to  go  back  (k)  is  determined  by  equating  the  time  interval  with  the  time  of  N  (number 
of  samples  of  past  location  and  velocity  information).  So  if  the  data  had  a  sampling  rate  of  5  per  second 
and  the  optimal  value  of  N  was  determined  to  be  10,  then  k  is  2  seconds 

This  velocity  value  is  then  averaged  with  all  the  previous  velocity  values  determined  over  the  past  k 
timeframe.  This  is  done  because  of  the  noise  in  the  data,  that  is,  if  the  data  was  clean  this  step  would  be 
eliminated.  The  estimated  velocity  is: 


Vyn  (S  Vyj  )  /  k 

So  it  is  this  value  of  velocity  that  is  used  in  the  prediction  equation: 

^  A  . 

y=y0+vynt 

The  following  graphics  (Figure  13  and  Figure  14)  provide  a  representative  depiction  of  the  algorithms 
success,  in  this  case  the  data  file  105-087  is  being  predicted.  This  particular  data  file  has  relatively  little 
noise  in  for  the  lineup  data  (Figure  13)  and  a  medium  amount  of  noise  in  the  glideslope  data  (Figure  14). 
The  predictor  performs  significantly  better  with  predictions  for  the  lineup  than  it  does  for  glideslope. 
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Figure  13.  Lineup  Prediction  for  Low-Noise  Data 
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Figure  14.  Glideslope  Prediction  for  Noisy  Data 


4.4.2.3  Comparison  and  Conclusion 

Out  of  all  the  investigations  conducted  regarding  the  development  of  aircraft  prediction  algorithms,  the 
two  most  successful  are  those  described  above,  ANFIS  and  Velocity.  However,  neither  of  these  two 
techniques  proved  better  overall.  That  is,  on  most  data  files  neither  technique  proved  to  be  statistically 
better.  There  were  individual  cases  where  one  technique  showed  better  predictions,  but  then  there  were 
counter  examples  where  the  other  technique  proved  better. 

Due  to  the  fact  that  the  task  consisted  of  solving  a  time  series  prediction  problem  in  which  only  past  and 
present  motion  profiles  are  provided,  it  is  surprising  that  the  predictions  are  as  good  as  they  are. 
Predictions  may  be  improved  if  environmental  conditions  were  provided,  such  as  the  wind  velocity;  also 
engine  settings  or  changes  in  engine  settings  could  aid  the  prediction,  even  if  this  information  was 
provided  only  second  hand  via  audible  cues  that  could  be  heard  on  the  deck;  finally  the  planes  attitude 
(pitch,  roll,  yaw)  and  changes  in  attitude  might  also  greatly  increase  the  predictions. 
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4.4.3  Prediction  of  Ship  Motion 
After  the  original  development  of  the  plane  prediction,  it  was  requested  that  a  ship  motion  prediction 
algorithm  be  investigated.  Unfortunately,  the  amount  of  information  provided  includes  only  past  ship 
pitch  and  roll  information  similar  to  the  plane  prediction  problem.  No  information  is  provided  about  sea 
state  or  other  environmental  conditions.  In  addition,  it  would  have  been  more  informative  if  the  actual 
three-dimensional  motion  profile  had  been  available  for  analysis. 

With  the  available  data  a  prediction  algorithm  based  on  sine  wave  projection  was  developed.  The 
prediction  4  seconds  into  the  future  is  shown  on  screen  via  a  flashing  of  an  extension  to  the  present  and 
past  pitch  and  roll  in  0.5  sec  intervals.  Continuously  displayed  lines  represent  the  current  and  the  past 
profile.  The  rightmost  end  of  the  lines  represents  the  current  pitch  and  roll  with  the  rest  of  the  line 
showing  the  past  pitch  and  roll. 


The  image  below  shows  the  ship’s  pitch  and  roll  display  with  the  predicted  pitch  and  roll  on. 


5  Storyboard  Evolution 

In  this  next  section  we  will  present  the  initial  storyboard  representation  and  latest  representation  of  the 
LSO  interface  produced  during  the  Phase  n.  In  this  way,  the  reader  will  be  able  to  see  how  the  display 
evolved  over  time,  and  will  help  the  readers  understand  how  the  interface  display  concepts  have  evolved 
throughout  this  project.  During  this  project  many  iterations  of  storyboards  were  created  with  the  specific 
intent  of  designing  displays  that  would  support  the  LSOs.  Because  the  interface  development  process 
involved  many  design  iterations  coupled  with  multiple  sessions  of  LSO  feedback,  it  would  be 
cumbersome  to  present  all  versions  of  the  design  concepts  in  this  report.  Also,  while  many  features  will 
be  discussed,  not  all  were  considered  critical  and  as  such  we  have  included  the  attached  PADAL  CD. 
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Figure  15.  An  Initial  Design 


Figure  15  depicts  one  of  the  initial  concepts  for  addressing  pilot  trending  in  the  display.  This  display 
should  be  conceptualized  in  four  distinct  areas:  The  left  two-thirds  of  the  display  represents  current 
aircraft  location  with  time  and  distance  considerations,  the  top  right-third  includes  pilot  information,  the 
middle  right-third  shows  pilot  trends,  and  the  bottom  right-third  gives  the  LSO  information  as  to  what  a 
“nice  correction”  would  be. 
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Figure  16.  Pilot  Trends  and  Flight  Information 


One  of  the  later  display  designs  is  shown  in  Figure  16  above.  It  consists  of  4  main  portions  and  the 
specifications  of  each  are  outlined  as  follows:  Incoming  flight  display  with  history  and  prediction  of 
future  aircraft  position  in  the  upper  left  two-thirds,  deck  motion  history  and  prediction  by  distance  from 
the  ramp  in  the  bottom  left-third,  pilot  trends  by  segment  in  the  upper  right-third,  and  pilot  information  in 
the  far  bottom  right.  Everything  displayed  to  the  right  of  pilot  trends  and  information  is  for 
demonstration  purposes  only.  (Note:  we  recommend  that  all  features  can  be  turned  on  or  off  as  the  user 
wishes.) 

5. 1  Potential  Sources  of  Data  to  Support  Display  Design  Recommendations 

While  the  storyboard  designs  were  being  developed  we  gave  considerable  attention  to  the  realities  of  the 
environment  and  domain  and  whether  we  would  be  able  to  acquire  the  data  necessary  to  implement  the 
display  concepts.  Figure  17  addresses  sources  of  data  required  for  the  Graphical  Representation  (upper 
left  portion  of  the  display  area),  and  provides  the  same  information  for  the  remaining  areas  of  the  display. 

While  we  recognize  that  much  of  the  data  we  will  need  to  implement  in  the  displays  are  already  available 
in  one  form  or  another,  we  feel  that  we  will  need  better,  more  consistent  data  sources  to  provide  us  some 
of  the  following: 

a)  Today's  weather 

b)  .  Today's  Case  I,  II,  or  HI 

c)  Past  Passes  Information  (Advanced  APARTS  database) 

d)  Similarity  of  current  pass  based  on  similar  condition  data 
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e)  Information  on  relative  importance  of  previous  recoveries. 
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Figure  17.  Data  sources:  Real-time  graphs 


5.2  LSO  Interface  Design  and  Implementation  for  Pilot  Trends 

Klein  Associates  applied  decision-centered  design  to  determine  the  best  display  options  for  rapid 
decision  support.  The  LSO  domain  is  fairly  unique  in  the  requirement  for  the  LSO  to  quickly  perceive 
the  likely  pattern  of  motion  (the  approach).  Thus  the  PADAL  system  must  rapidly  convey  relevant  past 
cases  of  approaches. 

The  interface  display  is  designed  primarily  for  the  CAG/Backup  LSO.  One  idea  is  to  take  as  much 
advantage  of  the  expertise  of  the  CAG/Backup  LSO,  to  support  this  individual’s  decision  making,  and  to 
then  permit  this  individual  to  make  the  judgment  as  to  whether  or  not  some  information  should  or  should 
not  be  passed  on  to  the  Controlling  LSO  during  actual  flight  operations. 

5.3  Evaluation 

Testing  and  evaluation  of  the  interface  concepts  occurred  throughout  the  development  process.  Klein 
Associates  was  the  lead  on  this  task  and  consulted  primarily  with  Ret.  Commander  Frank  Pfieffer  and 
multiple  LSO  instructors  at  the  LSO  School  in  Norfolk,  VA.  Klein  Associates  and  SHAI  visited  the  LSO 
School  on  many  occasions  and  talked  with  dozens  of  active  LSOs  about  the  interface  design,  focusing 
specifically  on  the  pilot  trending  aids.  We  received  feedback  during  each  of  the  trips  (two  separate  trials 
during  the  Phase  II  proper  and  one  trial  during  the  Option  period),  and  used  the  data  to  enhance  features 
of  the  interface.  The  evaluation  method  was  an  informal  process  and  is  described  below. 

5.3.1  LSO  Trials 

There  were  three  major  LSO  Trials.  Each  trial  was  conducted  using  an  informal  method  of  evaluation 
and  feedback.  The  process  entailed  interviewing  highly  experienced  LSOs  about  the  design  concepts 
followed  by  LSO  reactions  to  specific  features  of  the  design.  Each  LSO  that  took  part  in  the  trials  was 
instructed  to  run  through  the  display  demonstrations  we  had  built  for  them  and  to  subsequently  provide 
feedback. 

Klein  Associates  was  interested  in  collecting  feedback  on  many  different  aspects  of  the  display,  which 
included  factors  like  color,  size,  and  location  of  features  on  the  display.  More  importantly  though,  we 
were  interested  in  how  the  information  was  presented,  if  it  was  useful,  and  if  it  provided  support  for  their 
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decision-making.  The  first  two  trials  were  not  conducted  on  an  actual  simulator,  however,  the  LSOs 
were  able  to  “put  themselves  in  the  moment”  and  comment  on  potential  environmental  factors  that  would 
effect  the  use  of  the  interface.  They  identified  factors  like  sun  glare,  proper  lighting  at  night  so  as  not  to 
blind  them,  water  on  the  screen,  and  effective  use  of  colors  so  the  features  would  stand  out  during  a  quick 
glance  of  the  screen.  In  addition  to  these  more  peripheral  display  issues,  the  first  trials,  conducted  in 
December  1999  and  February  1999  at  the  LSO  School,  primarily  concentrated  on  information  produced 
by  pilot  trending.  And  the  second  trials,  conducted  in  September  1999  at  the  LSO  school,  primarily 
demonstrated  the  results  of  the  pilot  trending  research  and  interface  implementation.  Following  each 
LSO  Trial  the  software  was  enhanced  based  on  LSO  feedback  and  other  results  of  the  trials. 

The  third  LSO  Trial  was  conducted  with  the  PADAL  software  attached  and  reading  data  from  the  LSO 
Trainer.  Details  of  this  trial  are  provided  in  the  section  below. 

5.3.1. 1  LSO  Trial  of  the  PADAL  System  2000  December  20 

The  PADAL  system  was  evaluated  while  LSOs  waved  passes  in  the  LSO  Trainer  just  as  they  would 
during  normal  training.  The  PADAL  software  was  run  on  a  portable  PC  placed  where  the  HUD  normally 
is  located  so  that  it  would  be  part  of  the  LSO’s  normal  scan. 

Pre-Trial  Briefing 

On  Tuesday,  19  December  I  met  with  LCDR  Watkins,  the  OIC  of  the  LSO  School.  The  purpose  of  the 
meeting  was  to: 

•  provide  background  and  status  on  the  Piloted  Approach  Decision  Aid  Logic  System  (PADAL) 
project, 

•  familiarize  LCDR  Watkins  with  the  present  PADAL  software, 

•  outline  and  review  the  proposed  LSO  Trial  format,  and 

•  learn  about  the  quantitative  and  qualitative  criteria  utilized  to  evaluate  LSO  performance  (in 
the  LSO  Trainer). 

Participants 

The  following  Landing  Signal  Officers  participated  in  the  LSO  Trial  of  the  PADAL  Software. 

•  LCDR  Watkins 

•  LCDR  Snow 

•  LT  Lawrence 

•  LCDR  Burden 

•  LCDR  Gray 

•  LCDR  Bulis 

All  participants  are  highly  accomplished  Landing  Signal  Officers  and/or  Landing  Signal  Officer 
instructors.  This  situation  contrasts  with  previous  reviews  of  the  evolving  PADAL  system  that  consisted 
of  accomplished  LSOs  and  more  junior  LSOs.  As  is  the  case  with  decision  aids  in  any  domain,  it  is  the 
more  novice  users  that  will  benefit  the  most.  The  LSO  Trial  would  have  benefited  from  the  additional 
feedback  of  more  novice  LSO,  but  due  to  the  impending  downtown  of  the  LSO  Trainer  (Januaiy  2001)  a 
trial  prior  to  the  downtown  is  very  beneficial. 

Process  of  the  LSO  Trial 
1 .  Briefed  LSOs  on  the  functionality  of  PADAL 

The  PADAL  project’s  purpose  and  status  was  provided  to  all  participants.  After  this  the  software  was 
demonstrated  via  the  use  of  stored  data  of  real  data  from  actual  landings  on  aircraft  carriers.  During  the 
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demonstration  landings,  the  various  decision  aids  provided  by  the  software  were  introduced.  Each  of  the 
decision  aids,  including: 

•  current  aircraft  position  (red  dot), 

•  recent  position  ‘tail’  (gray  dots), 

•  future  position  (2  seconds  hence)  indicator  (blue  dot) 

•  velocity  vector  (green  line  emanating  from  aircraft  position  indicator), 

•  deck  status  indicator, 

•  glideslope  and  lineup  axes, 

•  threshold  circles,  and 

•  range  ladder. 

The  scales  used  for  the  glideslope  and  lineup  axes  where  described.  The  lineup  axis  uses  a  fixed  scale 
where  the  point  where  the  inner  (reasonable  deviation)  threshold  circle  crosses  the  lineup  axes  is  20  feet 
from  a  perfect  lineup  always  (that  is,  whether  the  plane  is  1  mile  from  the  deck  or  over  the  ramp). 

The  glideslope  uses  a  varying  or  ‘funneling’  scale.  At  1  nautical  mile  the  point  where  the  inner 
(reasonable  deviation)  threshold  circle  crosses  the  glideslope  axes  is  20  feet,  which  is  approximately  0.2 
degrees  above  the  perfect  glideslope.  As  the  plane  approaches  the  act  as  a  funnel  and  adjust  to  always 
show  the  approximate  0.2  degrees  above  the  perfect  glideslope;  that  is,  at  2  miles  this  would  be  40  feet, 
at  a  !4  mile  this  would  be  10  feet,  at  Va  mile  it  is  5  feet,  etc. 

2.  Waving 

The  LSOs  waved  just  as  they  would  wave  during  normal  use  of  the  LSO  Trainer,  with  a  controlling  LSO 
and  a  backup  LSO.  The  passes  occurred  under  simulated  poor  visibility  conditions,  such  as  night  and 
fog,  situations  where  the  likelihood  of  an  LSO  utilizing  a  decision  aid  is  maximal. 

Original  arrangement  for  Waving 

The  PADAL  software  was  placed  on  a  table  below  the  LSO  workstation  and  was  not  made 
available  during  an  initial  set  of  passes  (  ~  4)  for  the  controlling  LSO.  After  which  another  set  of 
passes  were  conducted  where  the  LSO  could  monitor  the  PADAL  software. 

The  initial  arrangement  was  found  to  be  unsatisfactory  as  the  LSOs  infrequently  look  away  from 
the  aircraft  in  any  case  and  now  with  the  extra  PADAL  software  the  normal  scan  process  was 
destroyed  or  at  least  seriously  hampered. 

Modified  and  Final  Arrangement  for  Waving 

Since  the  PADAL  decision  aids  are  not  intended  to  be  on  a  different  display,  but  incorporated 
into  the  controlling  display  it  was  decided  to  replace  the  heads-up  display  of  the  LSO 
workstation  with  the  PADAL  display  thus  placing  it  in  a  position  that  is  part  of  the  normal  scan. 
Another  change  made  at  this  juncture  was  to  always  make  the  PADAL  display  available  to  both 
the  controlling  and  backup  LSOs.  This  allowed  more  opportunity  to  judge  the  decision  aids 
because  of  the  relative  low-frequency  that  an  LSO  actually  looks  away  from  aircraft,  and  the  low 
possibility  of  improving  the  performance  of  this  set  of  LSOs  due  to  their  experience  and  the 
limitation  of  using  PADAL  in  its  current  state  for  critical  wave  off  decisions  (as  described 
below). 

At  the  Ramp  Funneling  Limitation  on  Glideslope  &  Wave-off  Point 


39 


Stottler  Henke  Associates,  Inc.  N68335-98-C-0027 

Due  to  the  tunneling  scale  used  for  the  glideslope  axis  (described  above)  PADAL  was  found  to 
be  a  poor  decision  aid  at  the  very  end  of  the  landing  pass,  starting  at  about  the  portion  of  the 
landing  where  the  critical  wave-off  decision  needs  to  be  made.  Prior  research  implied  that  the 
tunneling  scale  caused  a  problem  at  the  very  end  of  the  pass,  (since  very  small  deviations  in 
glideslope  are  magnified  in  PADAL  as  the  distance  approaches  zero),  but  after  the  critical  wave- 
off  point.  This  LSO  trial  showed  how  late  in  the  pass  the  wave-off  decision  can  occur,  a  lesson 
learned. 


Results 

After  all  the  LSOs  had  their  opportunity  to  wave  passes,  a  group  discussion  immediately  followed  to 
compare  reactions  to  the  various  decision  aids  available  in  PADAL.  There  was  an  overall  consensus 
regarding  most  of  the  decisions  aids.  The  reactions  to  each  of  the  decision  aids  will  be  discussed  below. 

1.  Current  aircraft  position  (red  dot) 

The  aircraft  position  indicator  should  be  the  easiest  entity  to  find  and  comprehend  on  the  screen. 
Questions  were  not  asked  directly  about  if  the  present  representation  was  deficient,  however, 
general  questions  were  repeatedly  asking  where  improvements  could  be  made.  No 
improvements  were  ever  offered  regarding  the  aircraft  position  representation,  in  addition  no 
criticism  was  ever  articulated  regarding  the  aircraft  position  symbol. 

Result:  no  improvements  need  to  be  made. 

2.  Recent  position  ‘tail’  (gray  dots) 

The  initial  overall  reaction  to  the  history  symbols  or  recent  position  tail  consisting  of  gray  dots 
(smaller  for  positions  further  in  the  past)  was  that  they  were  superfluous  and  simply  added  clutter 
to  the  screen.  However,  after  further  discussion  amongst  the  LSOs  it  was  concluded  that 
knowing  the  plane’s  histoiy  could  provide  benefit  when  the  plane  is  farther  out  then  At  The  Start 
(~  %  of  a  nautical  mile),  but  once  the  aircraft  was  At  The  Start  or  closer  the  LSO  would  have  a 
complete  mental  model  of  the  plane’s  histoiy  and  the  recent  position  tail  would  be  unneeded  and 
thus  added  to  clutter  and  could  be  detrimental. 

Result:  useful  while  the  aircraft  is  farther  then  At  The  Start  only;  (do  NOT  show  after  the  plane 
reaches  At  The  Start). 

3.  Future  position  (2  seconds  hence)  indicator  (blue  dot) 

The  essentially  unanimous  consensus  on  the  future  position  indicator  was  that  it  provided  no 
benefit,  and  in  many  cases  was  simply  confusing.. 

Result:  future  position  indicator  in  NOT  useful  and  may  be  confusing. 

4.  Velocity  vector  (green  line  emanating  from  aircraft  position  indicator) 

The  LSOs  concurred  that  the  velocity  vector  provided  the  most  benefit  of  the  decision  aids 
provided  in  PADAL.  The  velocity  vector  representation/symbology  was  discussed,  with  the 
concern  being  rapid  comprehension;  that  is,  could  a  modified  symbology  allow  the  LSO  to  more 
rapidly  discern  the  velocity. 

The  consensus  was  that  the  present  vector  representation  was  satisfactory. 
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Note  that  the  velocity  vector  represents  the  ‘velocity’  of  the  plane  in  the  lineup/glideslope  plane 
only,  it  does  not  consider  the  approach  speed  of  the  aircraft. 

Result:  lineup/glideslope  velocity  vector  is  useful. 

5.  Deck  status  indicator 

Most  LSOs  stated  that  they  did  not  refer  to  the  deck  status  indicator  provided  in  PADAL  because 
they  simply  referred  to  the  other  deck  status  indicators  provided  (outside  of  PADAL).  However, 
there  was  some  use  of  the  deck  status  indicator  by  the  backup  LSO. 

Result:  prominent  deck  status  indicator  may  be  useful  (especially  to  backup  LSO). 

6.  Glideslope  and  lineup  axes 

The  lineup  axes  scale  type  is  the  same  as  in  the  present  HUD,  which  was  fine  with  the  LSOs. 

The  glideslope’s  variable  ‘funneling’  scale  (as  described  above)  did  not  show  benefit  and  its 
problems  At  The  Ramp  contributed  to  a  negative  assessment.  Despite  the  problems  with  the 
glideslope’s  variable  ‘funneling’  scale  interest  was  expressed  for  an  improved  funneling  scale  for 
glideslope. 

Result:  use  traditional  scales  for  glideslope  and  lineup  axes  as  on  the  present  HUD. 

7.  Threshold  circles 

The  overall  concept  of  the  threshold  circles  was  found  to  be  useful  as  gages  or  at  least  non¬ 
distracting.  However,  most  confusion  with  their  use  or  utility  was  in  relation  to  the  glideslope 
funnel  based  scale. 

Result:  threshold  circles  are  beneficial. 

8.  Range  ladder 

None  of  the  LSOs  reported  referring  to  the  range  ladder  on  the  PADAL  screen.  This  was  not  due 
to  its  merit. 

Result:  Inconclusive 


6  Phase  II  Prototype  System 

This  section  describes  the  design  of  the  PADAL  Phase  II  prototype.  The  system  structure  and  major 
components  of  the  PADAL  Phase  II  prototype  are  detailed  below.  The  system  has  been  deployed 
utilizing  object-oriented  design,  the  C++  computer  language  on  Pentium  class  hardware  under  MS 
Windows  operating  systems.  More  information  may  be  found  in  the  User’s  Manual,  which  is  included  as 
Appendix  H. 

6.1  Display  Design 

The  display  panel  design  is  as  shown  in  Figure  18  below. 
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Figure  18.  Display  Panel  Layout 


It  consists  of  5  main  portions  and  the  specifications  of  each  are  outlined  as  follows.  This  is  displayed  on 
the  upper  left  comer  of  the  display  panel  and  can  be  seen  in  Figure  18.  The  features  in  the  display  are 
described  below  in  further  detail. 
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6.1.1  Incoming  graphical  display 
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Figure  19.  Incoming  Flight  Display 


Figure  19  is  roughly  the  2-D  equivalent  of  the  y-z  plane  of  the  3-D  trajectory  description.  A  typical 
incoming  flight  trajectory  in  3-D  is  as  shown  in  Figure  20.  The  center  of  the  3-D  axes  can  be  considered 
as  the  aircraft  carrier’s  center  of  gravity  (CG). 
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Figure  21.  Axes  and  circles 
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In  Figure  21,  the  horizontal  axis  represents  the  ideal  glideslope  based  on  the  Basic  Angle  in  effect  at  the 
time.  Ideally  this  angle  represents  3.5  degrees  up  and  out  from  the  deck  and  is  the  path  the  hook  should 
follow  to  touch  between  the  second  and  third  wires.  The  FRESNELL  Lens  on  board  ship  is  adjusted 
based  on  the  aircraft  type  to  match  the  particular  aircraft  hook-to-eye  value.  The  glide  slope  will  be  a 
basic  calculated  line  that  may  vary  some  with  the  specific  carrier  and  is  basic  angle  dependent. 

The  vertical  axis,  also  depicted  in  Figure  21,  represents  the  lineup  or  centerline  reference.  This  line  is 
the  actual  centerline  reference  for  the  angled  deck.  Exactly  how  the  location  of  the  aircraft  is  placed  with 
respect  to  this  line  will  be  determined  from  the  SPN-46  radar  data  and  the  dynamic  centerline. 

The  overall  frame  makes  up  about  60%  of  the  width  and  about  80%  of  the  height  of  the  display  monitor. 
It  is  situated  in  the  upper  left  comer  of  the  display  monitor.  This  feature  is  similar  to  what  the  LSOs  see 
on  the  PLAT  video,  a  screen  they  currently  use  to  determine  centerline  and  glideslope  deviations  as  the 
aircraft  approaches  the  deck. 

Deviation  Circles 

Two  concentric  ellipses  represent  the  thresholds  for  reasonable  and  gross  deviations.  The  area  bounded 
by  the  smaller  ellipse  corresponds  to  reasonable  deviation  and  the  area  outside  the  larger  ellipse 
corresponds  to  gross  deviation. 

In  the  knowledge  elicitation,  we  identified  specific  quantitative  deviations  that  LSOs  would  consider  to 
be  normal  or  gross  (see  Appendix  A,  data  specifics  piece),  based  on  each  segment  of  the  approach. 
Because  these  deviations  changed  per  each  segment,  the  size  of  the  circles  would  need  to  change  as  well. 
This  rationale  fed  into  our  initial  design  where  we  attempted  to  incorporate  dynamic  deviation  circles  as 
the  aircraft  approached  the  deck.  After  showing  this  aspect  to  many  LSOs,  the  prevailing  feedback  was 
to  leave  the  circles  static,  and  to  permit  the  LSOs  to  make  their  own  calculations  in  their  heads.  The 
LSOs  saw  circles  that  are  shrinking  and  increasing  in  size  as  more  of  a  distraction  than  as  a  support 
function. 

Range  ladder 

The  range  ladder  as  shown  in  Figure  22  provides  the  x-position  (distance  from  the  ship)  information  of 
the  aircraft.  The  range  ladder  moves  up  the  centerline  as  it  approaches  the  aircraft  carrier.  The  actual 
range  of  the  aircraft  is  indicated  at  the  crosshair.  For  example,  in  Figure  22  the  x-position  of  the  aircraft 
is  slightly  less  than  %  of  a  mile  from  the  ship. 
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Figure  22.  Range  ladder 

This  too!  is  continuously  moving,  showing  the  aircraft’s  location  from  the  ramp  in  roughly  quarter-mile 
increments.  The  IC  mark  is  1/8  mile  from  the  AR  mark,  IM  is  1/2  mile  from  the  AR  mark,  X  is  3/4  mile 
from  AR  the  mark,  and  1NM  is  1  mile  from  the  AR  mark.  Each  of  these  reference  marks  is  familiar  to 
the  LSOs  as  they  use  them  for  grading  segments  of  an  aircraft  approach.  It  should  be  noted  that  these  are 
relative  reference  points  and  are  not  regarded  as  absolute.  The  range  ladder  is  controlled  by  the  AR 
mark  on  the  ladder.  The  AR  mark  position  variable  is  initialized  to  be  at  the  bottom  of  the  vertical  axes  in 
Figure  18  and  remains  so  until  the  x-position  of  the  aircraft  is  smaller  than  1 .25  miles  and  then  begins  to 
move  dynamically  upward. 

The  position  of  the  range  ladder  has  changed  significantly  from  the  initial  design  concept  where  it  was 
seated  in  the  center  of  the  incoming  graphical  display  as  seen  in  Figure  15.  Many  of  the  LSOs  found  this 
to  be  distracting  and  also  commented  that  it  cluttered  the  screen.  Because  many  of  them  liked  the  feature 
so  much,  a  solution  was  to  move  it  to  the  left,  as  seen  in  Figure  1 8,  where  it  could  be  quickly  viewed  and 
unobstructed  from  other  dynamic  features.  The  range  ladder  still  utilizes  the  horizontal  axis  as  a 
reference  point  of  location. 

State  of  the  Deck 

Colored  markers  are  shown  in  the  range  ladder  to  indicate  the  state  of  the  deck.  The  position  of  the  mark 
on  the  range  ladder  is  an  indication  of  where  the  pilot  may  be  waved  off  if  the  status  of  the  deck  does  not 
change.  State  of  the  deck  was  not  considered  in  the  initial  design  concept  but  was  seen  as  an  important 
feature  to  include  after  frequent  discussions  with  the  LSOs.  In  addition,  many  of  the  latest  accidents  at 
sea  have  occurred  because  the  deck  was  not  clear  or  set  for  safe  landings.  Interestingly,  a  warning  does 
appear  on  the  current  LSO  workstation,  but  no  warning  exists  for  the  100  and  10  foot  standards  that  must 
be  met  for  clearing  a  foul  deck  and  a  deck  not  set  for  traps  (wires  are  not  retracted). 

•  A  red  mark  #—  indicates  a  foul  deck  (i.e.,  the  deck  is  not  ready  and  the  LSO  may  need  to 
wave  off  the  pilot  in  order  for  the  aircraft  to  clear  the  ramp  at  the  100-ft.  standard). 

•  An  amber  mark  indicates  the  equipment  is  not  set  for  the  trap.  It  shows  at  what  point  the  LSO 
must  wave  off  the  pilot  in  order  to  clear  the  10-ft.  ramp  clearance  standard. 
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•  The  amber  mark  will  turn  green  (i.e.,  ),  to  indicate  the  deck  is  clear  and  ready  for  landing. 

These  again  are  just  reference  points  and  are  not  absolute  values.  Many  other  factors,  like  aircraft  weight 
and  configuration  to  name  a  few,  affect  when  and  where  an  aircraft  can  be  waved-off.  Many  of  the  LSOs 
are  very  sensitive  to  having  a  computer  determine  where  a  wave-off  can  occur  without  taking  into 
consideration  the  many  other  factors  that  would  affect  the  clearance  of  the  100  and  10-foot  standards. 

Incoming  flight  position 

A  small  red  solid  circle  as  shown  in  Figure  23  indicates  the  current  position  of  the  aircraft.  The  position 
variable  is  governed  by  the  current  reading  of  lineup  and  glideslope  deviation  (taken  from  SPN-46  radar 
data).  This  is  permanently  displayed  on  the  panel  as  a  dynamic  feature.  This  is  a  critical  piece  of 
information  since  it  is  required  before  looking  at  any  trends  and/or  oscillations.  We  have  to  be  able  to 
track  where  the  aircraft  currently  is  to  be  able  to  do  any  of  these.  By  being  able  to  track  the  aircraft’s 
current  location  we  are  able  to: 

•  Identify  where  the  aircraft  has  been  in  the  past 

•  Identify  where  the  aircraft  is  with  respect  to  glideslope 

•  Identify  where  the  aircraft  is  with  respect  to  lineup  or  centerline 

•  Translate  this  information  (past  and  present)  into  pilot  trends  and  to  provide  history  for  the  entire 
pass. 


Figure  23.  Current  aircraft  position 


This  particular  feature  is  the  basis  for  all  items  listed  above.  We  intentionally  use  a  solid  circle  for 
several  reasons:  not  to  inadvertently  give  any  impression  of  knowledge  about  wing  position  (up,  down, 
etc.),  so  it  can  be  easily  recognized  on  the  screen  with  the  other  features,  and  so  that  it  shows  up  in  a  sun 
glare. 


Recent  aircraft  trajectory  profile 

To  display  the  history  of  what  the  aircraft  has  done,  seven  gray  dots  of  varying  sizes  are  displayed  on  the 
panel  to  represent  the  recent  7  positions  of  the  aircraft.  These  may  be  seen  in  Figure  18  and  Figure  19. 
An  array  of  7  variables  is  used  for  this  representation  and  each  member  of  the  array  corresponds  to: 


aircraft_proflle[0] 

aircraftjprofile[l] 

aircraft_profile[2] 

aircraft_profile[3] 

aircraft_profile[4] 

aircraft_profile[5] 

aircraft_profile[6] 


:  the  position  of  the 
:  the  position  of  the 
:  the  position  of  the 
:  the  position  of  the 
:  the  position  of  the 
:  the  position  of  the 
:  the  position  of  the 


aircraft  at  t  -35  sec. 
aircraft  at  t  -30  sec. 
aircraft  at  t  -25  sec. 
aircraft  at  t  -20  sec. 
aircraft  at  t  -15  sec. 
aircraft  at  t  -10  sec. 
aircraft  at  t  -5  sec. 


(smallest  gray  dot) 


(largest  gray  dot) 
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where  t  is  the  current  time. 

Starting  with  aircraft _profile[0],  then  aircraft_profile[l], ....  and ,  aircraft_profile[6],  each  is  displayed  on 
the  screen,  remaining  on  the  screen  for  about  two  seconds.  Many  variations  of  this  feature  were  talked 
about  with  the  current  design  being  the  most  comfortable  for  the  LSOs.  Many  were  concerned  with 
screen  clutter  and  the  dynamic  nature  of  the  design  being  distracting,  but  most  were  satisfied  with  the 
current  design.  The  LSOs  thought  that  the  history  could  be  picked  up  by  a  quick  glance  of  the  screen 
without  having  to  wait  long  for  the  history  to  be  displayed.  The  third  and  final  LSO  Trials  resulted  in  a 
the  result  that  the  history  dots  are  very  useful  while  the  aircraft  is  farther  then  At  The  Start;  but  should  no 
longer  be  shown  after  the  plane  reaches  At  The  Start. 

6.1.2  Pilot  Trending 
Pilot  information 

The  pilot  name,  aircraft  type/model,  and  side  number  are  displayed  in  the  middle  portion  of  the  display 
panel.  It  has  the  following  format: 

Pilot  name:  String  (e.g.,  Wilcox) 

Aircraft  type:  String  (e.g.,  F14B) 

Aircraft  side  number:  (e.g.,  103) 

This  is  just  the  basic  information  about  who  is  flying,  what  is  flying,  and  which  specific  aircraft  it  is  - 
all-important  for  the  LSO  to  know.  The  pilot’s  name  conveys  a  lot  of  information  to  the  LSO  when  it  is  a 
pilot  who  is  part  of  their  squadron  or  wing  and  they  have  been  deployed  long  enough  for  the  LSOs  to 
become  familiar  with  how  the  individual  flies.  If  it  is  during  carrier  qualifications  or  it  is  a  pilot  who  is 
not  part  of  their  unit  (i.e.,  COD  pilots,  etc.)  then  less  information  is  conveyed,  other  than  it’s  a  pilot  they 
do  not  know. 

The  other  information  shown  is  already  available  to  the  LSO  but  we  feel  that  is  doesn’t  hurt  to  repeat  it. 
The  main  concern,  however,  is  that  a  pilot  may  be  “swapped”  out  for  the  day’s  flight  and  what  is  shown 
is  not  accurate.  We  have  to  assume  there  will  be  some  way  to  verify  whom  the  pilot  actually  is. 
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Figure  24.  Example  of  Last  10  scatter  plot  trend 


Figure  24  displays  five  square  windows  with  cross-axis  line-up  in  a  column  to  the  right  of  the  incoming 
flight  graphical  display.  It  occupies  about  15%  of  the  total  screen  width  as  shown  in  the  Figure  18  above. 

Initially  we  wanted  to  see  how  the  aircraft  was  approaching  the  carrier  and  give  trends  for  the  approach 
linguistically,  based  on  how  this  particular  approach  started.  However,  the  LSOs  asked  that  we  just  show 
the  pilots’  regular  trend  under  the  particular  conditions  for  the  current  pass  (e.g.,  night  landings  in  F/A- 
18B).  They  have  also  asked  that  pilot  trends  be  flashed  up  all  at  once  as  opposed  to  sequentially  as  the 
aircraft  passes  each  segment  of  the  approach.  The  reason  for  the  transformation  to  a  scatter  plot  was  due 
to  the  feedback  received  from  the  LSOs.  Many  of  them  found  the  grading  and  comments  cumbersome  to 
read,  and  many  generally  like  the  look  of  the  scatter  plots. 
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Trend  analysis  can  provide  the  LSO  with  some  very  useful  information.  However,  in  order  to  guard 
against  the  possibility  that  the  pilots  "trend",  or  average,  might  equate  to  a  zero  and  give  the  LSO  no 
trend  information  at  all,  we  have  inserted  a  scatter  plot  for  specific  sections  of  the  approach  (at  the  start, 
IM,  IC,  AR,  and  I/OW).  In  Figure  24,  each  scatter  plot  displays  the  pilot’s  last  ten  passes  under  similar 
conditions  (night,  case  3,  etc.).  The  current  flight  position  is  indicated  by  a  relatively  large  round  red  dot, 
the  rest  of  the  ten  recent  dots  have  varying  sizes  indicating  their  recency.  The  most  recent  one  is  largest. 

The  scatter  plots  allow  the  LSOs  to  examine  what  a  pilot  has  done  in  their  last  ten  passes,  which  in  some 
instances  can  give  them  more  information  than  just  the  "trend."  In  addition,  it  allows  the  LSO  to  quickly 
identify  deviations  in  a  pilot’s  past  performance  and  match  them  up  with  what  is  currently  happening. 

As  the  pilot  enters  each  section  of  the  pattern,  the  scatter  plot  identifies  approaches  from  passes  that 
closely  match  the  current  position  of  the  aircraft.  From  this  point,  the  LSO  is  able  to  follow  these  circles 
in  each  subsequent  scatter  plot.  It  is  important  to  be  able  to  follow  these  circles  because  the  purpose  of 
the  plots  is  to  give  the  LSO  a  picture  of  how  this  pilot  looked  in  sequence  as  they  made  the  approach  to 
the  ramp  during  past  approaches. 

The  Last  10  column  is  also  shared  by  a  Similar  10  feature  and  APARTS  trend  data.  The  Similar  10 
feature  was  developed  using  CBR  and  is  another  way  to  represent  pilot  trends.  Although  it  is  more 
complicated  than  the  Last  10,  the  Similar  10  concept  is  based  on  the  initial  idea  of  how  the  aircraft  is 
approaching  the  carrier,  and  to  give  trends  for  the  approach  based  on  how  this  particular 
approach  started.  This  is  a  useful  feature  when  a  pilot,  usually  more  experienced,  flies  many  passes.  If 
s/he  always  overshoots  the  start,  the  LSO  can  look  at  the  trend  data  on  the  scatter  plot  to  see  what  s/he 
normally  does  in  the  subsequent  segments  of  the  pass. 

The  LSOs  saw  the  APARTS  data  as  extremely  useful  because  they  use  the  tool  all  the  time.  The 
APARTS  database  is  a  tool  that  the  LSOs  use  to  enter  grading/comments  for  pilots,  and  has  its  own 
trending  system  based  on  day  or  night  landings. 

6.2  Interface  to  LSO  Simulator 

PADAL  has  been  designed,  implemented  and  tested  to  interface  with  the  LSO  Training  Simulator.  That 
is,  PADAL  can  receive  its  data  from  stored  information  on  disk  or  can  receive  live  pass  information  via 
ethemet  and  receive  data  from  the  LSO  Training  Simulator. 

7  Serendipities 

The  following  results  of  the  effort  may  prove  very  valuable  to  the  Navy  even  though  they  were  never 
specifically  requested  in  the  contract. 

7.1  On-Board  Debriefing  Tool 

LSOs  have  requested  a  playback  capability  for  debriefing,  review,  and  instructional  purposes  that  could 
be  used  on-board  an  aircraft  carrier.  Requested  features  in  the  playback  version  include  pause,  random 
playback  at  any  stage,  and  slow/fast  play.  This  capability  has  been  requested  at  all  the  LSO  Trials  and 
during  the  PADAL  Q&A  session  at  the  OAG  meeting. 

This  playback  capability  is  very  close  to  already  being  part  of  the  present  software.  The  requested 
features,  such  as  pause,  random  playback  at  any  stage,  and  slow/fast  play,  are  already  part  of  the 
software. 
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The  only  piece  missing  is  interfacing  with  the  data  available  on  the  aircraft  carrier,  which  is  unknown  at 
this  time  since  the  VISUAL  system  output  specification  has  not  been  completed  to  date.  However,  since 
the  software  already  can  read  from  multiple  Navy  sources  including  the  LSO  Trainer  it  should  be  a  minor 
modification  to  allow  for  reading  from  VISUAL  data  once  it  is  available. 

7.2  VISUAL  or  HUD  display  for  LSO  Trainer  during  transition 

In  order  for  SHAI  to  develop  the  various  ideas  that  were  tested  in  the  PADAL  software  throughout  the 
Phase  II;  the  software  had  to  be  designed  as  a  rapid  development  testbed  for  LSO  Station  concepts. 

Since  this  rapid  development  testbed  is  part  of  the  software  architecture  it  is  relatively  easy  to  modify  the 
PADAL  software’s  interface. 

One  valuable  use  may  be  to  modify  PADAL  to  provide  a  VISUAL  or  HUD  display.  Why? 

So  that  a  (portable)  PC  could  be  used  throughout  the  transition  of  the  LSO  Trainer  from  the  HUD 
to  VISUAL,  initially  to  provide  the  option  of  showing  the  VISUAL  screen  when  the  HUD  is  at 
the  LSO  School  and  then  to  provide  the  option  of  showing  the  HUD  after  VISUAL  is  installed 
and  the  HUD  is  still  in  service  on  aircraft  carriers. 

When  both  the  HUD  and  VISUAL  are  active  in  the  fleet,  the  LSO  School  could  train  on  a  complete  HUD 
or  VISUAL  system,  while  the  other  system  could  at  least  be  represented  via  a  PC  showing  the 
appropriate  (HUD  or  VISUAL)  display.  That  is,  initially  when  the  HUD  is  still  at  the  LSO  School  and 
the  fleet  starts  to  introduce  VISUAL  to  the  fleet,  students  from  VISUAL  aircraft  carriers  could  be  trained 
while  waving  with  a  replica  of  the  VISUAL  display  via  the  modified  PADAL  on  a  portable  PC  screen. 
When  the  HUD  at  the  LSO  School  is  replaced  by  VISUAL,  students  from  HUD  aircraft  carriers  could  be 
trained  while  waving  with  a  replica  of  the  HUD  display  via  the  modified  PADAL  on  a  portable  PC 
screen. 

Another  benefit  may  be  in  finding  problems  with  visual  representations  in  the  VISUAL  display.  That  is, 
since  it  is  much  more  rapid  to  create  the  VISUAL  interface  by  modifying  PADAL,  the  VISUAL  interface 
can  be  evaluated  in  the  LSO  Trainer  sooner.  Thus  feedback  regarding  the  interface  in  this  environment 
can  occur  sooner. 

8  Deliverables 

The  PADAL  CD  contains: 

•  PADAL  Software,  and  the 

•  PADAL  User’s  Manual. 

9  Phase  III  and  Future  Work 

There  are  enormous  opportunities  for  the  Department  of  Defense  to  make  use  of  the  results  of  this  effort, 
both  directly  and  indirectly.  Most  directly,  the  LSO  is  the  direct  target  for  this  effort.  Furthermore,  the 
ability  to  safely  direct  the  landing  of  helicopters  is  an  important  capability  for  almost  all  Naval  platforms. 
Many  of  the  techniques  used  to  develop  the  PADAL  System  can  be  applied  to  other  platforms  as  well. 

As  shown  in  the  following  Figure  25,  a  portion  of  the  PADAL  Phase  I  and  Phase  II  research  will  be 
incorporated  into  the  VISUAL  system  and  thus  the  LSO  workstation  software,  running  on  the  LSO’s 
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workstation  hardware.  In  particular,  portions  of  the  technology  developed  for  PADAL  has  already  been 
incorporated  into  the  VISUAL  system. 


Figure  25.  PADAL  Context 


Besides  the  contributions  PADAL  has  already  made  to  the  VISUAL  system,  there  are  important  areas 
where  Phase  m  work  could  add  to  these  contributions. 

1)  Provide  PADAL  in  a  continuous  training  environment 

Allow  for  continuous  LSO  Trials.  That  is,  a  working  PADAL  would  remain  with  the  LSO  Trainer 
(the  interface  has  been  completed  during  Phase  II  work).  This  task  would  allow  for  more  realistic 
testing  of  the  various  PADAL  components  to  better  judge  there  utility  and  thus  to  determine  which 
components  should  be  integrated  with  VISUAL.  Users  would  have  the  option  to  view  or  not  view 
PADAL.  LSO  would  provide  feedback  after  using  the  entire  trainer  and  note  whether  they  ever  used 
PADAL,  and  if  so  what  was  found  to  be  useful.  A  final  test  would  be  to  remove  PADAL  from  the 
LSO  Trainer  environment  after  many  LSOs  had  become  familiar  with  it  and  measure  reaction  to  its 
absence. 

2)  Test  PADAL  at  sea  or  the  carrier  landing  practice  field 

Conduct  a  LSO  Trail  on  an  aircraft  carrier  or  at  the  carrier  landing  practice  field,  in  a  non-obstructive 
manner. 

3)  Cone  for  showing  prediction  of  ship 


Continue  to  develop  the  ship-motion-prediction  algorithm  to  provide  a  cone  of  prediction.  It  is 
difficult  to  predict  the  four  second  hence  deck  location  due  to  the  limited  information  provided  for 
prediction,  however,  the  prediction  could  be  enhanced  by  providing  a  cone  of  prediction  as  shown  in 
Figure  26.  The  thick  black  line  is  the  past  motion  with  the  present  position  shown  at  the  rightmost 
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side  of  the  line;  the  prediction  cone  shows  the  range  of  possible  locations  the  deck  may  take  during 
the  next  4  seconds.  The  cone  would  be  much  more  descriptive  than  the  example  shown,  for  it  could 
be  color  shaded  to  represent  the  likelihood  of  the  deck  being  in  a  certain  part  of  the  cone. 

4)  Improve  plane-prediction  algorithm  (with  more  data) 

Continue  to  develop  the  plan-prediction  algorithm  with  more  complete  data  including  environmental 
conditions.  This  development  is  critically  dependent  on  more  landing  data.  With  more  landings  the 
present  algorithms  can  be  further  tested  and  enhanced.  In  addition,  the  prediction  can  be  modified 
from  a  point  prediction  to  a  range  prediction  where  the  future  location  of  the  plane  is  represented  to 
show  a  region  where  the  plane  will  most  likely  to  be.  This  is  similar  to  the  proposed  modification  to 
the  ship-motion  prediction. 

5)  Offline  version  with  playback  capability 

LSOs  have  requested  a  playback  capability  for  debriefing,  review,  and  instructional  purposes. 
Requested  features  in  the  playback  version  include  pause,  random  playback  at  any  stage,  and 
slow/fast  play.  It  is  also  preferred  that  the  offline  version  is  network  accessible  with  password 
protection  in  different  levels. 

This  capability  has  been  requested  at  all  the  LSO  Trials  and  during  the  PADAL  Q&A  session  at  the 
OAG  meeting.  This  playback  capability  could  be  first  implemented  via  the  LSO  Trainer,  such  that 
whenever  the  system  is  attached  to  the  LSO  Trainer  all  passes  would  be  recorded  for  later  playback; 

6)  Ramp  motion  alarm 

LSO  requested  an  alarm  if  the  ramp  motion  exceeds  8  ft/second;  also  alarm  if  ramp  motion  ‘changes 
significantly’.  The  specifics  of  change  significantly  would  need  to  be  elicited  from  the  LSO 
community. 

7)  Adaptively  learn  ranges  corresponding  to  LSO  shorthand  comments 

The  ranges  of  deviation  corresponding  to  the  various  LSO  shorthand  comments  where  determined 
from  knowledge  elicitation  from  various  LSOs,  and  implemented  using  fuzzy  system  approach.  The 
actual  ranges  may  be  different  in  actuality  than  as  described  on  dry  land  and  the  ranges  may  change 
over  time  and  under  different  conditions. 

PADAL  could  be  modified  to  adaptively  learn  and  update  the  fuzzy  system  for  determining  the 
appropriate  ranges  that  correspond  to  the  LSO  shorthand  comments.  The  information  used  to  learn 
the  ranges  could  also  be  used  to  graphically  depict  via  a  scatter  plot  the  actual  aircraft  location  versus 
LSO  comments.  This  could  be  used  as  a  training  and  review  tool. 

8)  Unified  Aircraft  Glideslope,  lineup  and  ship  deck  pitch  display  with  predictions 

In  the  Figure  27,  the  gray  areas  are  present  to  indicate  a  different  scale.  These  indicate  2  miles,  while 
the  entire  remainder  of  the  display  concentrates  on  that  last  mile.  As  we  have  been  doing,  we  would 
still  provide  predictions  for  the  a/c  glide  slope,  lineup  and  the  deck  motion  (shown  in  red  in  the 
Figure  27).  So  we  get  histoiy,  actual,  and  predicted  for  these  three.  The  focus  would  be  to  take  this 
initial  integrated  display  concept  and  hone  it  to  create  one  integrated  display  that  provides  valuable 
information  and  can  be  quickly  comprehended. 
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In  addition  to  the  LSO  specific  applications,  the  advanced  motion  case  display  technology  developed 
for  LSOs  would  be  useful  in  domains  where  past  cases  of  motion  patterns  can  be  retrieved  and  when 
this  historic  data  must  be  quickly  conveyed  to  the  user.  One  example  is  threat  assessment  in 
complicated  tactical  scenarios  (especially  those  related  to  defense)  such  as  those  faced  by  the  E2C 
officer,  the  AAWC  (Anti- Air  Warfare  Coordinator)  in  the  CIC,  and  the  Weapons  Director  aboard  an 
AWACS.  SHAI  has  worked  in  all  of  these  fields. 

10  Conclusions 

This  report  has  summarized  the  tasks  and  results  of  this  Phase  II  SBIR  project  (including  the  Option). 

The  project  utilized  artificial  intelligence  and  cognitive  task  analysis  to  develop  a  LSO  driven  decision 
support  tool. 

The  project  determined  the  significant  aircraft  approach  parameters  and  similarity  measures  and 
important  pilot  considerations  and  similarity  measures.  From  this  information  the  project  developed  pilot 
trending  techniques  and  software  using  case-based  reasoning  and  combinations  of  other  AI  techniques. 

In  addition,  in  conjunction  with  many  LSOs,  the  project  determined  the  best  display  options  and  most 
appropriate  display  logic  for  the  information  produced  by  the  pilot  trending  modules,  and  designed  and 
implemented  the  resulting  LSO  interface. 
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same  same  Same  same  Also  can  check  same 

See  note  [15]  closing  speed  from 

HUD  to  determine  if 

Configuration  of  aircraft  as  acft  is  below  max 

well  as  sequence/separation  HI  trap  ground  speed. 


Stottler  Henke  Associates,  Inc.  Final  Report  Appendix  A 


Configuration  of  aircraft  as  No  appr  light  No  appr  light  No  appr  light  On  or  off  W/O  for  fly  by  gear 

well  as  sequence/separation  III _ '  _  check 
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Appendix  B:  Corollary  Grid  Notes 

The  numbers  can  be  found  in  certain  cells  within  the  DRTs  in  Appendix  A.  To  save  room  in  the  tables  we 

have  decided  to  add  a  description  in  this  section  that  corresponds  to  those  numbers. _ 

Number  Description _ 

1  _ Kts:  Nautical  Miles  Per  Hour _ _ 

2  _ CCA:  Carrier  Controlled  Approach _ 

3  AOA:  Angle  of  Attack _ 

4  CCA  calls:  If  not  in  EMCON  and  if  aircraft  is  getting  voice  command  form  CATC  controller.  Also, 

LSO  may  not  be  on  pilot’s  frequency  at  this  point  since  Case  III  are  DUAL  FREQUENCY.  For  Case 
III,  a  different  frequency  is  set  for  every  other  aircraft:  aircraft  101  is  on  freq.  A,  aircraft  102  is  on  freq. 

_ B,  aircraft  103  is  on  freq.  A,  etc... _ 

5  Approach  light  on  and  steady  means  that  gear  is  down.  Approach  light  steady  means  that  hook  is 
down.  Approach  light  flashing  means  hook  is  up.  However,  at  night,  if  one  of  the  three  approach  lights 
( red  =  fast,  amber  =  on  speed,  green  =  slow)  are  burned  out,  and  this  is  the  light  that  should  be 
illuminated  given  the  aircraft  speed  at  the  time,  then  the  appearance  will  be  that  the  gear  are  up  (e.g.,  no 
approach  light).  In  this  case  the  LSO  will  ask  the  pilot  to  “show  me  as  FAST”,  “show  me  as  SLOW”, 
or  “show  me  as  an  APPROACH  light.”  The  pilot  will  momentarily  dip  his  nose  or  pull  up  slightly  in 
order  to  get  the  AOA  to  change  enough  to  illuminate  one  of  the  other  two  lights  to  confirm  gear  down. 

_ If  no  lights  working,  a  W/O  is  required  with  a  fly  by  to  confirm  gear  down  or  not. _ 

6  Discrimination  ability:  Case  I  and  II  -  Line  up  at  the  135E  position  (e.g.  rate  of  turn  which  results  in  a 
line  up  outcome)  is  a  function  of  perceiving  rate  of  turn,  ship’s  track,  relative  wind  and  turn  rate 
corrections  by  the  pilot.  At  this  point  little  attention  and  little  LSO  input  can  be  made.  Discrimination 

_ is  gross  and  difficult  to  quantify. _ _ 

7  _ Fpm:  feet  per  mile _ 


8 

At  1  NM,  if  visibility  permits,  pilots  will  begin  seeing  the  ball  and  visual  line  up  cues.  Most  will 
include  this  information  in  their  scan  of  instruments.  This  is  a  critical  time  for  pilots  in  that  most  will 
perceive  themselves  as  high  on  the  G/S  at  this  “first  look”  at  the  ship  and  tend  to  settle  below  the  G/S. 
Difficult  for  the  pilot  to  see  anything  at  1  NM. 

At  1  NM  last  aircraft  should  have  trapped  and  next  aircraft  will  be  on  the  LSO’s  frequency  and  HUD 
display. 

10 

The  difference  between  discrimination  ability  is  a  function  of  perceptive  ability  primarily  at  night. 

11 

Case  I  and  II  for  Attitude/Speed:  At  the  90E,  pilot  will  see  the  result  of  his/her  approach  turn  from 
180Eto  the  90E  and  will  make  most  of  the  major  roll  corrections  from  the  90E  to  X.  These  changes  in 
AOB  require  A/S  changes  to  maintain  on  speed.  These  changes  are  difficult  to  accurately  see. 

12 

Case  III  for  Attitude/Speed:  In  addition  to  simply  fast  or  slow,  and  more  critical,  is  the  circumstances  of 
a  pilot  who  has  not  been  able  to  resolve  his  speed  after  2  miles  of  descent  on  the  G/S.  Average 
performance  is  consistently  a  little  fast,  poor  is  >  8  kts  fast,  consistently  slow  or  unstable  slow  to  on 
speed  to  fast  to  on  speed  to  slow  equals  lots  of  problems. 

13 

Case  I  and  II  for  Line  up  at  90E:  A  turn  requiring  >  45E  AOB  is  difficult  at  best.  However,  a  shallow 
turn  of  <  15E  AOB  will  necessitate  a  rapid  increase  to  over  45E  AOB  to  stop  the  aircraft  on  the  C/L  at 
the  X  and  is  therefore  a  more  critical  error/correction  association. 

14 

G/S  &G/P  all  cases:  Although  a  nose  down  (ND)  action  by  the  pilot  would  appear  to  be  an  A/S 
correction,  IM  -  AR  is  a  quick  G/P  correction  to  increase  ROD.  In  fact,  it’s  quicker  than  waiting  for  a 
power  reduction  to  affect  sink  rate.  Improper,  but  not  uncommon,  it  is  somewhat  acceptable  but  can 
have  disastrous  consequences  if  not  properly  executed  and  corrected. 

15 

All  specific  cues  are  amplified  by  aircrafts  relative  size  and  peripheral  environment  from  IM  -  AR. 

16 

However,  very  rare  since  aircraft  are  in  level  flight  prior  to  G/S  intercept  at  200  ft.  This  is  basic 
instrument  flying  at  this  point. 
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Additional  Notes 


<  Case  I  and  II  use  primary  frequency  A.  Case  HI  uses  dual  frequency. 


100/B 

“trapped” 


RAMP 


The  controller  is  talking  to  101/A  and  103/A.  When  ball  is  called,  controller  stops  talking  to  103/A  for 
about  20  seconds,  and  the  LSO  does  not  stop  talking.  Before  this,  LSO  gets  G/S  information  from  the 
controller’s  calls. 

<  The  new  guy  is  easiest  to  control  because  of  heightened  expectation  of  problem. 

<  For  estimation  of  airspeed,  actual  perception  is  the  degree  of  attitude,  which  translates  to  kts. 

<  HUD  not  available  @  2  NM.  Only  when  aircraft  is  IC  - 1  NM  (unless  1st  aircraft  is  in). 

<  At  the  90E,  G/S,  attitude  and  L/U  are  starting  to  get  more  and  more  “mashed”  together. 

<  ND  will  cause  a  settle  and  this  is  want  we  want  to  avoid. 

<  Notice  that  as  pilots  get  closer  to  the  ramp,  the  Top  pilot  has  a  possible  -  probable  problem  IC-AR. 
The  reason  for  this  is  because  LSO  doesn’t  expect  Top  pilot  to  be  out  of  parameters  here.  If  he  is, 
he  may  have  more  problems.  If  Top  guy  has  problems  here,  he  becomes  more  like  the  FNG. 

<  No  approach  light  check  required  during  daytime  because  LSO  can  get  a  visual. 
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Appendix  C:  SPN46  Track  Data 

Header  Contains: 


Label 

Pass  Number 

Channel 

Acft  Side  No. 

RECORD 

00056 

A 

105 

15:41:14 

DATA  CONTAINS: 


TIME 

(mm:ss) 

since 

lockon 

X 

position 

(ft) 

I 

Z 

position 

(ft) 

Ship’s 

pitch 

(deg) 

Ship’s 

roll 

(deg) 

Closing 

speed 

(ft/sec) 

Sink 

speed 

(ft/sec) 

00:44.4500 

2866 

-37.50 

220 

.27466 

.70862 

-218 

-6.40625 

Aircraft  Types  vs.  Side  Numbers 


F-14A 

F-14B 

F-18 

A-6 

C-2 

101 

210 

301 

520 

COD 

202 

105 

502 

205 
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Appendix  D:  Comment  Examples  and  Explanations 


I 

Q> 

& 


i 

I* 

1 


PtH 


V  p  o 
£ 
8 

■3)  °  b 

•r-T  "M  £ 

Men 


a  3 


O  u 

S  |  a 

Ph  >_)i  q 


S3 

§  g 


»  § 

r  \ 


§  1  I 

s  8  S 

g  «  a 

^  HH  ^ 

n  8  i 


R 


|  R  S  g 

I  I  S  g 


iESA  TMRDOT  \X  LOIM  ND/IC  FBAR  Not  enough  straight  away.  Too  much  rate  of  descent  out  of  the  turn.  Fly  down  through  glide  path  at  the  start. 

_ _ Low  in  the  middle.  Nose  down,  fly  up  through  the  glide  path  in  close.  Fast  and  flat  at  the  ramp. _ 

IEPQLOX)  LOSLOIM _ Not  enough  power  and  slightly  low  at  the  start  Very  low  and  slow  in  the  middle. _ 

JG  (LOX)  /IM  OCHIC  \AR  LL  Long  in  the  groove.  Slightly  low  at  the  start.  Fly  up  through  the  glide  path  in  the  middle.  Over  control  and 

high  in  close.  Fly  down  through  the  glide  path  at  the  ramp.  Land  Left. 


Stottler  Henke  Associates,  Inc. 


Final  Report  Appendix  E 


Appendix  E:  Landing  Profile 


SQUADRON  VF- 103  LANDING  PROBLEM  PROFILE 

RECOVERY  PERIOD  8/19/1995  11:15:00  PM  -  10/20/1997  12:30:00  PM 
PILOT  SELECTED  DOMINO 


AIRCRAFT  ;  ALL 
DAY/NITE/ALL  A 
MOVLAS  s  A 


GLIDESLOPE  DESCENT  RATE  SPEED  POWER  ATTIT.  LINEUP 

HI  LO  TM  NE  FAST  SLOW  (  +  )  (-)  +  WING 


X 


IM 


jfcfc  ; fife  5ft 

IC 


###### 

###### 

###### 

###### 


TOTAL  NUMBER  OF  APPROACHES  -  4  LEGEND:  HIGH  ###### 

MEDIUM  —  — 
LOW  - 
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Appendix  F:  Deviations  Table 


Deviations  Table 

Acceptable,  little  or 
no  correction 
needed 

Reasonable,  correction 
will  be  needed 

Gross  Deviation 

1  Mile 

High  (ft) 

No  Limit 

No  limit 

No  limit 

Low  (ft) 

25 

50 

100 

Fast  (nmph) 

5 

10 

20 

Slow  (nmph) 

5 

10 

20 

Left  (ft) 

>150 

Right  (ft) 

EfMM MHMH 

>150 

3/4  Mile  (near  the  Start  “X” 

High  (ft) 

25 

50 

Low  (ft) 

12.5 

25 

Fast  (nmph) 

5  (Case  1  &  2) 

2.5  (Case  3) 

10  (Case  1  &  2) 

5  (Case  3) 

20  (Case  1  &  2) 

10  (Case  3) 

Slow  (nmph) 

2.5 

5 

Left  (ft) 

37.5 

75 

150 

Right  (ft) 

37.5 

75 

150 

V4-1/2  Mile  (In  Middle  “IM”) 

High  (ft) 

20 

40 

80 

Low  (ft) 

10 

20 

40 

Fast  (nmph) 

5 

10 

20 

Slow  (nmph) 

2.5 

5 

10 

Left  (ft) 

25 

50  Case  1) 

50+  (Case  2  &  3) 

100 

Right  (ft) 

25 

100 

High  (ft) 

10 

20 

40 

Low  (ft) 

5 

10 

20 

Fast  (nmph) 

2.5 

5  or  >5  change 

Slow  (nmph) 

2.5 

5  or  >  5  change 

10 

Left  (ft) 

12.5 

25  or  >5°  AOB 

50 

Right (ft) 

12.5 

25  or  >5°  AOB 

50 

At  the  Ramp  (“AR”) 

High  (ft) 

5 

10  or  >200fpm  ROD  off 
ideal 

20 

Low  (ft) 

2.5 

5  or  >200fpm  ROD  off 
ideal 

10 

Fast  (nmph) 

>1.5°  pitch 

>3°  pitch 

>6°  pitch 

Slow  (nmph) 

>1.5°  pitch 

>3°  pitch 

>6°  pitch 

Left  (ft) 

7.5  or  >1.5°  AOB 

15  or  >3°  AOB 

30 

Right  (ft) 

7.5  or  >1.5°  AOB 

15  or  >3°  AOB 

30 

AOB  =  Angle  of  Bank  ROD  =  Rate  of  Descent 
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Appendix  G:  Summary  of  Variables  and  Formula 


variables 

units 

interpretation 

X 

ft. 

the  x  position  of  the  aircraft 

y 

ft. 

the  y  position  of  the  aircraft 

Z 

ft. 

the  z  position  of  the  aircraft 

closing  speed 

ft/sec 

dx/dt,  the  rate  of  change  of  x  with  respect  to  time 

sink  rate 

ft/sec 

dz/dt,  the  rate  of  change  of  z  with  respect  to  time 

lineup 

ft. 

This  is  the  deviation  of  the  y-coordinate  of  the  aircraft  from  the 
actual  centerline  reference  for  the  landing  (angled)  deck 

glideslope 

deg. 

tan'1  ( z/x),  nominal  glideslope  is  3.5° 

1NM 

mile 

1  mile  from  the  ramp  of  the  ship 

X(The  Start) 

approximately  l/2~  3/4  mile  from  the  ramp  of  the  ship 

IM(In  the 
Middle) 

approximately  1/4  mile  from  the  ramp  of  the  ship 

IC(In  Close) 

approximately  1/8  mile  from  the  ramp  of  the  ship 

AR(At  the 
Ramp) 

right  at  the  ramp  (stem  of  the  ship) 
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Contract  Number  N68355-98-C-0027 
Agency:  Naval  Air  Warfare  Center  Aircraft  Division  (Lakehurst) 


Stottler  Henke  Associates,  Inc.  (SHAI) 
San  Mateo,  CA 


PADAL  User’s  Guide 


INTRODUCTION 


About  the  Document 

This  document  is  a  guide  to  using  the  Piloted  Approach  Decision  Aid  Logic  (PADAL)  system.  The 
document  provides  installation  instructions  as  well  as  a  reference  guide. 

Conventions 

This  document  uses  the  following  formatting  styles  to  indicate  special  text: 

Fixed  width  font  Represents  text  as  it  appears  on  the 

screen,  or  as  you  must  type  it. 

Bold  font  Represents  specific  instructions  or 

important  notes. 

Italics  Represents  key  words. 


PADAL  Background 

PADAL  (Piloted  Approach  Decision  Aid  Logic)  assists  the  Landing  Signal  Officers  (LSO)  in  making 
better  judgments.  PADAL  assists  the  LSO  by  utilizing  pilot  and  aircraft  information  and  utilizes  case-base 
reasoning  technology  to  retrieve  similar  cases.  The  pilot  information,  similar  flight  patterns,  and  flight 
position  predictions  are  displayed  on  the  panel  for  the  LSO’s  reference. 

Guiding  aircraft  to  land  on  aircraft  carrier  is  extremely  difficult.  The  LSO  observes  the  incoming  flight 
pattern  and  the  environmental  conditions  to  issue  proper  advice  to  guide  the  landing  of  the  aircraft.  Should 
a  wave-off  be  necessary  to  prevent  a  mishap  the  LSO  will  do  so  anytime  before  the  aircraft  gets  to  the 
ramp  of  the  aircraft  carrier.  The  LSO  grades  each  pilot  landing  with  an  overall  score  and  comments  for 
different  stages,  i.e.,  the  start  (X),  in  the  middle  (IM),  in  close  (IC),  at  the  ramp  (AR).  Comments  are  also 
given,  though,  implicitly  for  the  stages  before  1  NM  and  over  the  wire.  These  scores  and  comments 
affiliated  with  each  pilot  in  different  flights  are  stored  in  a  database  called  APARTS. 

Hardware  Requirements 

PADAL  is  designed  to  operate  on  a  PC  running  MS  Windows  95,  98  NT,  Windows  200  or  greater.  The 
program  will  require  a  minimum  of  15  megabytes  of  hard  drive  space.  The  program  is  optimally  displayed 
in  SVGA  mode  of  800  X  600  resolution  and  256  (or  greater)  colors. 
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Summary  of  Variables  and  Abbreviations 
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Installation 


Insert  the  PADAL  distribution  CD-ROM  into  the  drive,  the  system  may  perform  an  autorrn  starting  the 
installation  process  automatically,  if  this  does  not  occur  locate  the  setup .  exe  in  the  top  directory  of  the 
CD-ROM  and  double-click  to  start 

When  the  setup .  exe  program  is  starting  you  may  see  the  following  splash  image. 

** Ml  ■ 
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In  this  window  you  can  optionally  specify  a  different  path,  where  PADAL  should  be  installed,  by  pressing 
the  Browse  button. 

If  the  Current  Settings  displayed  in  this  window  are  correct,  press  the  Next  button  to  let  PADAL  Setup 
copy  all  required  files  to  the  specified  directory.  Once  copying  is  complete  the  installation  process  will 
finish  and  exit. 
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STARTING  PADAL 


gagacs 

To  start  the  PADAL  program,  select  the 

Programs 


menu,  then  select 


then  the 


Padal 

submenu,  and  finally  the 

Padal 

option.  This  will  launch  PADAL. 


Tutorial 


Starting  the  PADAL  program 


Start  the  PADAL  program  by  selecting  the  1 
submenu,  and  finally  the  Padal  option. 

Selecting  and  Starting  a  Simulation 


I  menu,  then  select  Programs,  then  the  Padal 


ir  Data  Source: -r  If 


Click  on  the 


SPN  46 

|  is  not  selected  in  the  Data  Source  menu,  select  SPN  46. 


button,  the  following  form  will  open. 
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If  0101-392  is  not  selected  in  the  SPN4 6  Data:  menu,  then  choose  it  from  the  SPN4 6  Data:  menu. 


Now  simply  click  on  I 


I  to  initiate  a  pilot  trending  demonstration.  The  PADAL  program  will 


delay  for  a  few  seconds  while  the  SPN46  data  is  being  loaded. 

Once  the  simulation  has  commenced,  many  properties  of  the  landing  are  presented  on  the  display  as 
described  below. 


Monitoring  a  Landing  Simulation 


Lineup  and  glideslope  deviation  profile 

The  largest  graphic  region  indicates  the  current  aircraft  position  with  #  the  symbol.  The  vertical  line 
represents  the  on  centerline,  while  the  horizontal  line  represents  the  on  glideslope.  In  the  example  image 
below  the  aircraft  is  just  left  of  the  perfect  lineup  (from  the  LSO’s  perspective)  and  just  a  little  below  the 
ideal  glideslope. 

The  two  circles  mark  the  boundaries  between  three  sections.  The  section  inside  the  inner  circle  represents 
deviations  permitted  for  a  perfect  or  OK  pass  grade.  The  section  from  the  inner  to  the  outer  circle 
constitutes  reasonable  deviations.  The  section  beyond  the  outer  circle  represents  gross  deviations. 


Recent  Positions  and  Predicted  Future  Location 

During  the  simulation,  a  sequence  of  7  gray  dots  is  displayed.  The  least  recent  Gray  dot  is  displayed  the 
smallest  and  the  dot  size  grows  as  the  dots  become  more  recent,  with  the  7th  dot  being  the  most  recent  and 
largest.  Thus  the  enlarging  Gray  dots  reflect  the  recent  track  history.  (Note  that  even  though  they  are  all 
shown  simultaneously  in  the  image  below,  they  are  drawn  sequentially). 

The  blue  circle  symbol,  ,  #  shows  a  prediction  of  where  the  plane  may  be  in  2  seconds. 
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Range  ladder 

The  range  ladder  on  the  left-hand  side  of  the  display  provides  distance  information  for 
the  aircraft.  The  range  ladder  moves  up  the  centerline  as  it  approaches  the  aircraft 
carrier.  The  actual  range  of  the  aircraft  is  indicated  at  the  crosshair.  Take  for  example, 
1  mi  the  figure  below  the  aircraft  is  slightly  less  than  %  of  a  mile  from  the  ship. 

The  colored  markings  signify  the  status  of  the  landing  deck  with  respect  to  obstacles. 

5 : x .  Both  the  red  and  the  amber  marks  will  appear  if  the  deck  is  not  ready  for  landing,  i.e., 


,c  Once  the  deck  is  cleared  of  obstacles  the  red  “  1 00ft”  marker  will  disappear  but  the 

fife"*  “10ft”  marker  will  remain  amber  until  all  equipment  (e.g.,  lens,  gear)  is  set  properly. 

Once  the  amber  marker  turns  green,  i.e., 

4ft- 

- AR 

9 

the  deck  is  clear,  and  all  the  equipment  is  set. 

Pilot  and  Aircraft  Information 

Relevant  information  of  the  pilot  of  the  incoming  flight  is  displayed  as 


Where  the  first  line  displays  the  pilot’s  name,  the  second  line  the  aircraft  type,  and  the  third  line  the 
aircraft’s  side  number. 
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The  lower  box  shows  the  total  number  of  traps  and  passes  by  this  particular  pilot.  The  line,  labeled 
Similar  Conditions  displays  how  many  traps  this  pilot  has  had  under  environmental/day/night  conditions 
similar  to  the  current  one. 


Last  10 

The  most  recent  10  cases  from  the  APARTS  database  will  be  displayed  in  the  subwindows  here.  The 
subwindows  correspond  to  stages  X,  IM,  IC,  AR,  and  O/W,  respectively. 

The  current  flight  position  is  indicated  by  a  relatively  large  round  red  dot.  The  other  dots  display  the 
pilot’s  last  ten  most  recent  passes,  the  ten  recent  dots  have  varying  sizes  indicating  their  recency.  The  most 
recent  one  is  the  largest. 

Each  scatter  plot  displays  the  pilot's  last  ten  passes  under  similar  conditions  (night,  visibility,  etc.). 

The  scatter  plots  allow  recall  what  a  pilot  has  done  in  their  last  ten  passes,  quickly  identifying  deviations  in 
a  pilot's  past  performance  and  juxtaposing  them  with  what  is  currently  occurring. 


84 


PAPAL  User’s  Guide 


PAPAL  User’s  Guide 


User  Options 


Start  or  Pause/Resume 

The  Start  or  Pause  buttons  may  be  pressed  at  anytime,  except  the  few  seconds  data  is  being  loaded  at 
the  beginning  of  a  simulation.  Press  the  Pause  button  while  the  simulation  is  running, 


notice  that  the  simulation  pauses  at  its  current  state;  also  notice  that  the  formerly  Pause  button  is  now  a 
Re  s ume  button. 

Press  the  Resume  button  and  notice  the  simulation  continues. 


Jumping  to  Sections  of  the  Approach 

The  buttons  shown  below  jump  to  the  beginning  of  the  designated  section  immediately  (once  a  simulation 
has  started).  The  buttons  may  be  used  to  jump  forward  or  backwards  from  the  present  position  in  the 
landing  simulation. 

Press  the  buttons  in  the  following  order  while  the  simulation  is  running,  AR,  IC,  IM,  X,  1NM. 


Waveoff 

The  Waveoff  button  may  be  used  to  signify  a  waveoff. 


Press  the  Waveoff  button,  the  entire  screen  flashes  with  a  reddish  background  while  retaining  all 
information.  The  screen  will  look  similar  to  the  following  image  after  the  Waveoff  button  has  been 
pressed. 

Press  the  Waveoff  button  again  to  stop  the  waveoff. 


86 


PAPAL  User’s  Guide 


Clear  for  Landing 

The  Clear  for  Landing  button  may  be  used  to  signify  that  the  aircraft  carrier  is  clear  to  accept  a 
landing. 


While  both  the  red  and  the  amber  marks  are  displayed  during  a  landing  simulation. 


press  the  Clear  for  Landing  button.  Notice  that  the  red  and  the  amber  marks  are  replaced  one  green 
mark, 


- AR 

> 

signifying  the  aircraft  carrier  is  clear  to  accept  a  landing.  Press  the  Clear  for  Landing  button  and 
notice  that  the  red  and  the  amber  mark  replace  the  green  mark. 

Quitting  the  software 

Either  click  on  the  Exit  button  on  the  main  display  window  or  hit  Esc  on  the  keyboard  to  quit  the 
PADAL  program. 
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Using  PADAL 

To  start  the  PADAL  program,  select  the  fllllll]  menu,  then  select  Programs  then  the  Padal  submenu, 
and  finally  the  Padal  option.  This  will  launch  PADAL. 

The  initial  PADAL  screen  will  appear  as  shown  in  the  Starting  PADAL  chapter. 

Padal  may  receive  its  input  from  three  sources.  Live,  which  means  PADAL  is 
connected  to  the  LSO  Simulator  at  Oceana  NAS  (or  a  compatible  data  source)  via 
an  ethemet  connection.  SPN  46  which  refers  to  data  files  stored  in  the  SPN  46  data 
format..  Replay,  which  refers  to  files  that  have  been  saved  via  the  ‘Save  Live  Data 
For  Replay’  option  in  the  Select  parameter  values  form. 


•  SPN46  Data 

This  fM *  ffiy* '  drop-down  list  provides  a  selection  of  the  different  aircraft  landings 

1  6eC  |  - .  .3a  -IC' .  .  j  available  from  the  SPN46  dataset.  The  choice  from  this  list  will  only 

ilIBi 
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be  used  if  the  SPN  46  has  been  selected  as  the  Data  Source. 


•  Replay: 

This  drop-down  list  provides  a  selection  of  the  different  aircraft  landings 
available  from  the  Replay  dataset.  The  choice  from  this  list  will  only  be 
used  if  the  Replay  has  been  selected  as  the  Data  Source.  Selecting  the 
“Save  Live  Data  for  Replay”  option  creates  a  Replay  file.  This  drop  down 
menu  shows  previously  recorded  data  sets. 


I Cortazar_M ay21  _1 806_  ▼  ! 


The  crosshair  in  the  largest  portion  of  the  main  screen  (the 
Trajectory  Display)  is  a  coordinate  system  with  the  vertical  axis 
representing  the  glideslope  deviation  from  the  nominal  value,  and 
the  horizontal  axis  representing  the  lineup  value.  The  nominal 
glideslope  value  is  normally  3.5°.  This  is  the  glideslope  of  an 
aircraft  when  it  appears  anywhere  along  the  horizontal  axis  in  the 
Trajectory  Display.  This  nominal  angle  may  be  defined  separately 


for  each  stage.  To  change  this  value,  enter  a  different  value  in  the  box  labeled  for  the  desired  stage. 


•  Velocity  &  Acceleration  Vector  Display 


Selections  providing  the  option  to  display  the  velocity  and  acceleration 
vectors.  The  No  vector,  option  will  suppress  the  display  of  the 
vectors.  By  not  checking  the  Static  Display  option  the  vectors  will 
be  shown  emanating  from  the  aircraft  locator  (the  red  circle).  If  the 
Static  Display  option  is  checked  then  the  acceleration  and  velocity 
vector  will  be  shown  in  the  upper  right  comer  of  the  Trajectory  Display. 


•  Glideslope  Deviation  Axis  Range 


The  Glideslope  Deviation  Axis  Range  defines  the  range  of  aircraft  glideslope 
deviation  angles  in  degrees  from  the  nominal  glideslope.  An  aircraft  with  a 
glideslope  deviation  angle  within  the  specified  range  will  appear  in  the  Trajectory 
Display..  The  default  range  of  the  vertical  axis  is  -1.5  degrees  on  the  lower  side 
and  1.5  degrees  on  the  upper  side,  these  values  are  user  modifiable. 


•  Lineup  Axis  Range 


The  Lineup  Axis  Range  defines  the  range  of  aircraft  lineup  values.  An  aircraft  with 
a  lineup  within  the  specified  range  will  appear  in  the  Trajectory  Display.  The 
default  range  of  the  horizontal  axes  is  from  -200  ft.  on  the  left  to  200  ft.  on  the  right, 
these  values  are  user  modifiable. 


•  1NM  Threshold 

The  NATOPS  manual  on  LSO  grading  and  symbology,  uses  the  following  three  terms:  OK  pass, 
reasonable  deviations,  and  gross  deviations.  Two  concentric  elliptical  closed  curves  divide  the  area  into 
three  sections: 

Inside  the  inner  ellipse/circle  are  the  deviations  permitted  to  receive  the  OK  pass  grade. 
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Variations  from  the  inner  to  the  outer  circle  constitute  reasonable  deviations. 

Deviations  beyond  the  outer  circle  represent  gross  deviations. 

The  lineup  and  glideslope  deviation,  in  feet,  is  chosen  for  the  INautical  Mile  range.  The  glideslope 

threshold  value  is  then  scaled  appropriately  for  the 

.  C  j- — .  |-LL threshold  to  remain  constant  (in  terms  of  angular 

deviation)  throughout  the  landing  pass.  Gross  deviation 
:  Lineup:  |20  j|  ^  ,  ;)20  j|?  threshold  is  computed  by  scaling  the  reasonable  deviation 

:  ■  threshold  by  a  factor  of  5 . 

Therefore  the  OK  pass  circle  (inner  circle)  has  a  radius  of  20  feet  or  diameter  of  40  feet  at  1  Nautical  mile, 
and  the  reasonable  deviation  circle  (outer  circle)  has  a  radius  of  100  feet  or  diameter  of  200  feet  at  1 
Nautical  mile. 

20  feet  radius 


•  Lineup  Prediction  Method 

The  Lineup  Prediction  Method  defines  the  prediction  algorithm  to 
be  used  for  the  aircraft’s  lineup  forecast.  There  are  4  options 
available,  ANFIS,  Constant,  Linear  and  Velocity,  for 
predicting  Lineup  2  seconds  in  the  future.  One  should  chose  either 
Velocity  or  ANFIS.  The  two  other  algorithms  provided  are  included  for  developmental  purposes. 


•  Glideslope  Prediction  Method 


The  Glideslope  Prediction  Method  defines  the  prediction  algorithm 
to  be  used  for  aircraft’s  glideslope  forecast.  There  are  4  options 
available,  ANFIS,  Constant,  Linear  and  Velocity,  for 
predicting  Lineup  2  seconds  in  the  future.  One  should  chose  either 


Velocity  or  ANFIS.  The  two  other  algorithms  provided  are  included  for  developmental  purposes. 


•  Raw  Data 

Due  to  the  noise  in  many  of  the  data  files,  a  filtering  function  has  been  implemented  so  that  the  plane 

EilillllllP  symbol  does  not  show  this  noise  on  the  screen.  By  selecting  the  Raw  Data  option  no 
Raw  Data,  filtering  will  occur  and  the  aircraft  location  will  be  displayed  on  screen  using  the  raw 
data  from  the  input  file. 


•  Prediction  Display  Freq. 

The  Prediction  Display  Frequency  slider  sets  the  display  frequency 
for  the  prediction  flight  position.  The  prediction  remains  unchanged 
in  that  it  is  always  a  prediction  2  seconds  into  the  future;  this  slider 
simply  allows  the  setting  of  how  often  this  prediction  is  shown  on 
the  screen,  the  setting  range  is  from  1/10  of  a  second  to  2  seconds. 


•  Log 
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Selecting  the  Log  option  produces  a  text  log  file  of  the  data  that  appears  in  the  Trajectory 
|p||h!jg|  Display  as  well  as  supplemental  information  such  as  predicted  lineup,  predicted  glideslope 
and  other  variables.  This  option  is  normally  used  for  development  purposes  only. 


•  Save  Live  Data  for  Replay 

,  Selecting  this  option  saves  the  landing  information  to  a  date  stamped  file 

Date  For  If|l|f|  for  later  Replay.  This  is  normally  used  when  the  data  source  is  Live,  since 
this  operation  would  be  redundant  in  the  case  a  data  file  already  exists. 
The  difference  between  this  option  and  the  Log  is  that  this  option  records  only  flight  and  ship  information. 


Clicking 

aborts  the  form.  If  OK  was  selected,  the  form  will  close  and  a  display  similar  to  the  image  below  will  be 
displayed. 

Note:  It  may  take  2  or  more  seconds  to  load  the  data  before  the  simulation  starts. 


istarts  the  simulation  with  using  the  current  settings,  while  clicking 
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Display  Panel 

The  entire  display  panel  is  as  shown  below. 


The  above  screen  shot  shows  all  of  the  main  screen’s  display  components.  The  display  is  divided  into 
three  primary  areas.  The  upper  left  half  of  the  display  is  the  incoming  flight  graphical  display.  The  bottom 
portion,  under  the  incoming  flight  graphical  display,  may  be  configured  to  display  ship  motion  or  aircraft 
trajectory-related  LSO  comments.  Clicking  on  the  appropriate  “Trend”  option  in  the  lower  right  comer  of 
the  panel  makes  this  selection.  In  the  display  panel  image  shown  above,  the  ship  motion  display  option  is 
chosen.  The  area  to  the  right  of  the  incoming  flight  graphical  display  provides  information  by-segment 
(i.e.,  the  Start,  In  the  Middle,  etc.)  showing  a  scatter  plot  of  the  pilot's  last  10  recoveries  under  these  types 
of  conditions,  and  below  this  basic  aircraft  and  pilot  information.  The  following  sections  discuss  the 
screen  in  detail. 
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Incoming  Flight  Graphical  Display 

The  Incoming  Flight  Graphical  Display  is  the  largest  port  of  the  Display  Panel  occupying  the  upper-left 
portion  of  the  Display  Panel.  An  example  Incoming  Flight  Graphical  Display  is  shown  below. 


The  display  shows  a  graphical  representation  of  what  the  aircraft  is  actually  doing.  The  graphic  quickly 
shows  where  the  aircraft  is  located,  in  real  time,  with  respect  to  the  glideslope  and  the  centerline.  The  two 
crosshairs  in  this  display  are  always  fixed  with  the  intersection  of  "on  glideslope"  and  "on  centerline". 
The  Incoming  Flight  Graphical  Display  consists  of  many  components,  they  include: 

Glideslope  and  Lineup  Axes  and  Threshold  Circles, 

Recent  Aircraft  Trajectory  Profile  and  Predicted  future  position, 

Range  Ladder  with  State  of  Deck, 

Deck  Decorator. 

These  components  are  further  described  in  the  following  sections. 

Aircraft  Location,  Glideslope/Lineup  Axes  and  Threshold  Circles 

The  Aircraft  Location,  Glideslope/Lineup  Axes  and  Threshold  Circles  components  of  the  Incoming  Flight 
Graphical  Display  looks  as  shown  below. 
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Aircraft  Location 

The  aircraft’s  current  (x,z)  position  is  indicated  9  by  the  symbol  (red  circle).  A  circle  was  selected 
to  reduce  the  possibility  of  the  LSO  interpreting  it  as  any  indication  of  the  status  of  the  wings  (level,  up, 
down). 

Glideslope/Lineup  Axes 

The  horizontal  axis  represents  the  lineup  deviation  from  the  ship’s  perspective.  As  usual,  the  left  side  and 
right  side  of  the  horizontal  axis  correspond  to  negative  and  positive  lineup  deviation,  respectively. 

The  vertical  axis  represents  the  glideslope  deviation  with  upper  axis  and  lower  axis  representing  positive 
and  negative  glideslope  deviations  respectively. 

The  glideslope  deviation  =  glideslope  -  nominal  glideslope  (default  nominal  glideslope  is  3.5  °). 
Threshold  Circles 

In  the  NATOPS  manual  on  LSO  grading  and  symbology,  they  use  the  following  three  terms:  OK  pass, 
reasonable  deviations,  and  gross  deviations.  These  terms  are  the  basis  for  the  two  concentric  elliptical 
closed  curves  shown  in  this  screen.  Two  concentric  elliptical  closed  curves  divide  the  area  into  three 
sections: 

Inside  the  inner  circle  are  the  deviations  permitted  to  still  get  the  OK  pass  grade. 

Variations  from  the  inner  to  the  outer  circle  constitute  reasonable  deviations. 

Beyond  the  outer  circle  represents  gross  deviations. 

Reasonable  deviation  threshold  radii  may  be  set  through  the  parameter  value  selection  dialog  box  that  pops 
up  when  the  “Start”  button  is  pressed.  The  lineup  and  glideslope  deviation,  in  feet,  is  chosen  for  the 
INautical  Mile  range.  The  glideslope  threshold  value  is  then  scaled  appropriately  for  the  threshold  to 
remain  constant  (in  terms  of  angular  deviation)  throughout  the  landing  pass.  Gross  deviation  threshold  is 
computed  by  scaling  the  reasonable  deviation  threshold  by  a  factor  of  5. 
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Recent  Aircraft  Trajectory  Profile  and  Predicted  Position 


Recent  Aircraft  Trajectory  Profile 

Past  aircraft  positions  are  displayed  by  seven  gray  round  dots,  •  ,  of  increasing  size.  A  strobing  effect  is 
created  by  displaying  •  from  the  smallest  (most  distant),  to  the  largest  (most  recent)  .  In  so  doing,  the 
tendency  of  the  aircraft  can  be  easily  discerned.  All  seven  should  sequentially  "strobe"  in  about  1  to  1 
second.  They  repeat  the  strobe  about  every  3  seconds. 

The  first  (smallest)  dot  plots  the  aircraft’s  location  approximately  28  seconds  prior  to  the  current  position, 
the  second  dot  to  flash  shows  the  location  24  seconds  earlier,  the  third  shows  20  seconds  earlier,  the  fourth 
shows  16  seconds  earlier,  the  fifth  shows  12  seconds  earlier,  the  sixth  shows  8  seconds  earlier,  and  the 
seventh  (last)  shows  the  aircraft’s  location  4  seconds  earlier. 


Predicted  Future  Position 

The  predicted  future  position  of  the  aircraft  2  seconds  in  the  future  is  indicated  £  by  the  symbol 
(blue  circle). 

Note  that  if  the  predicted  future  position  and  the  present  position  are  close  enough  that  the  circles 
representing  each  overlap,  the  present  position  symbol  occludes  the  prediction  symbol.  For  example,  in 
the  image  below  the  predicted  future  position  symbol  overlaps  the  current  position  symbol  thus  the 
prediction  position  symbol  is  partially  occluded  by  the  present  position  symbol. 
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Range  Ladder  with  State  of  Deck 


Range  Ladder 

Two-dimensional  displays  do  not  show  the  distance  of  the  aircraft  from  the 
carrier.  To  provide  this  information,  the  range  ladder  is  displayed  on  the  left 
portion  of  Incoming  Flight  Graphical  Display.  As  the  aircraft  approaches  the 
aircraft  carrier,  the  range  ladder  moves  up  the  horizontal  axis,  (i.e.,  the  lineup), 
thus  providing  a  dynamic  reference  as  to  the  aircraft’s  location.  The  actual 
range  of  the  aircraft  is  indicated  at  the  intersection  of  the  ladder  with  the 
horizontal  axis,  (i.e.,  the  lineup).  In  the  graphic  the  aircraft  is  At  The  Start. 

The  rungs  mark  the  following  distances: 

IC  1/8  of  a  nautical  mile, 

IM  14  of  a  nautical  mile, 

X  %  of  a  nautical  mile, 

lmi  1  nautical  mile. 


-Imi 


-X 

-IM 


- AR 


State  of  the  Deck 

Colored  markers  are  shown  in  the  range  ladder  to  indicate  the  state  of  the  deck.  The  position  of  the  mark 
on  the  range  ladder  is  an  indication  of  where  the  pilot  may  be  waved  off  if  the  situation  does  not  change. 

•  A  red  mark  #—  indicates  a  foul  deck,  i.e.,  the  deck  is  not  ready  and  the  LSO  may  need  to 
wave  off  the  pilot  in  order  for  the  aircraft  to  clear  the  ramp  at  the  100-ft.  standard. 

•  An  amber  mark  indicates  the  equipment  is  not  set  for  the  trap.  It  shows  at  what  point  the 
LSO  must  wave  off  the  pilot  in  order  to  clear  the  10-ft.  ramp  clearance  standard. 

•  The  amber  mark  will  turn  green,  i.e.,  ,  to  indicate  the  pilot  is  in  a  position  to  clear  the  ramp 

by  10  ft. 

In  the  image  below,  the  colored  red  marker  indicates  a  foul  deck  (and  the  amber  marker  indicates  the 
equipment  is  not  set  for  the  trap). 


-lmi 


-X 

-IM 


- AR 


In  the  image  below,  the  green  marker  indicates  the  deck  is  clear  and  ready  for  landing. 
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Deck  decorator 

A  deck-like  decorator  is  provided  at  the  bottom  of  the  aircraft  motion  display  window.  Its  purpose  is  to 
provide  a  visual  queue  only,  and  to  provide  another  indicator  of  the  deck  status.  Note  it  does  NOT  reflect 
the  ship’s  motion  because  LSO  feedback  indicated  that  showing  the  ship’s  motion  proved  distracting. 

A  red  deck-like  decorator  indicates  the  deck  is  foul. 


A  green  deck-like  decorator  indicates  the  deck  is  clear. 
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Ship  Pitch  and  Roll  Display 


The  bottom  portion,  under  the  incoming  flight  graphical  display,  displays  ship  motion.  The  aircraft 
carrier’s  pitch  and  roll  are  represented  as  deviations  from  the  neutral  position,  which  is  shown  as  a 
horizontal  line  running  along  the  vertical  centerline  of  the  graphic  pane.  The  rightmost  end  of  each  curve 
shows  the  current  pitch  and  roll. 


Past  and  Present  Pitch  and  Roll 

Continuously  displayed  lines  represent  the  current  and  the  past  profile.  The  rightmost  end  of  the  lines 
represents  the  current  pitch  and  roll  with  the  rest  of  the  line  showing  the  past  pitch  and  roll.  The  default 
colors  are  red  for  pitch  and  black  for  roll.  These  colors  may  be  changed  by  selecting  from  the  drop-down 
lists  to  the  left  of  the  pitch/roll  display. 


Predicted  Pitch  and  Roll  (4  seconds  into  the  future) 

The  predicted  pitch  and  roll,  4  seconds  in  the  future,  is  flashed  on  and  off  in  0.5  sec  intervals.  The  image 
below  shows  the  ship’s  pitch  and  roll  display  with  the  predicted  pitch  and  roll  on. 


Pilot  Information 

Pilot  information  for  the  pilot  of  the  incoming  pass  is  displayed  in  the  lower  center  of  the  display  panel.  It 
has  the  following  format: 

Pilot  name  (e.g.,  Cortazar) 

Aircraft  type  (e.g.,  F123Z) 

Aircraft  side  number  (e.g.,  256) 

The  lower  box  shows  the  total  number  of  traps  and  passes  by  this  particular  pilot.  The  line,  labeled 
Similar  Conditions  displays  how  many  traps  this  pilot  has  had  under  environmental/day/night  conditions 
similar  to  the  current  one. 


98 


PAPAL  User’s  Guide 


Total  Traps:  94 
Similar  Conditions:  0 

ililfli 


The  Last  10  Profile 

Trend  analysis  can  provide  the  LSO  with  some  very  useful  information.  However,  in  order  to  guard  against 
the  possibility  that  the  pilots  "trend",  or  average,  might  equate  to  a  zero  and  give  the  LSO  no  trend 
information  at  all,  we  have  inserted  a  scatter  plot  for  specific  sections  of  the  approach  (at  the  start  (X),  IM, 
IC,  AR,  and  over  the  WIRE).  Each  scatter  plot  displays  the  pilot's  last  ten  passes  under  similar  conditions 
(night,  case  3,  etc.). 

The  scatter  plots  allow  the  LSOs  to  examine  what  a  pilot  has  done  in  his/her  last  ten  passes,  which  in  some 
instances  can  give  them  more  information  than  just  the  "trend."  In  addition,  it  allows  the  LSO  to  quickly 
identify  deviations  in  a  pilot's  past  performance  and  match  them  up  with  what  is  currently  happening. 

(Less  than  10  passes  will  be  shown  in  situations  where  10  passes  under  similar  conditions  are  not 
available.) 

As  the  pilot  enters  each  section  of  the  pattern,  the  scatter  plot  identifies  the  last  ten  passes  under  similar 
environmental  conditions.  From  this  point,  the  LSO  is  able  to  follow  these  circles  in  each  subsequent 
scatter  plots.  It  is  important  to  be  able  to  follow  these  circles  because  the  purpose  of  the  plots  is  to  give  the 
LSO  a  picture  of  how  this  pilot  looked  in  sequence  as  s/he  made  the  approach  to  the  ramp  during  past 
approaches. 

The  current  flight  position  is  indicated  by  a  relatively  large  round  red  dot,  the  rest  of  the  ten  recent  dots 
have  varying  sizes  indicating  their  recency.  The  most  recent  one  is  larger  as  shown  below. 
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The  Similar  10  Profile 

The  user  can  also  select 
Similar  10 

from  the  dropdown  list.  In  this  situation,  the  10  most  similar  cases  under  similar  environmental  conditions 
(night,  visibility,  etc.)  from  the  APARTS  database  will  be  displayed  in  the  subwindows.  The  subwindows 
correspond  to  stages  X,  IM,  IC,  AR,  and  O/W,  respectively. 

Just  as  in  the  Last  10,  the  current  flight  position  is  indicated  by  a  relatively  large  round  red  dot.  The  other 
dots  display  the  pilot’s  ten  most  similar  passes,  the  ten  recent  dots  have  varying  sizes  indicating  they’re 
similar.  The  most  similar  one  is  the  largest. 

The  scatter  plots  allow  recall  what  a  pilot  has  done  in  their  most  similar  ten  passes,  quickly  identifying 
situations  where  even  the  most  similar  passes  vary  greatly  from  the  present  pass,  revealing  a  situation  that 
the  pilot  has  never  seen  previously. 

As  the  pilot  enters  each  section  of  the  pattern,  the  scatter  plot  identifies  similar  approaches  that  most 
closely  match  the  current  position  of  the  aircraft.  From  this  point,  the  LSO  is  able  to  follow  these  circles 
in  each  subsequent  scatter  plot.  It  is  important  to  be  able  to  follow  these  circles  because  the  purpose  of  the 
plots  is  to  give  the  LSO  a  picture  of  how  this  pilot  looked  in  sequence  as  s/he  made  the  approach  to  the 
ramp  during  past  approaches. 
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Animation  Speed: 


{Normal 


Very  Fast 
Fasl 


Normal 


(Slow 
Very  Slow 


The  animation  speed  option  sets  the  speed  that  a  SPN46  Data  or  Replay  data  is 
processed.  The  Normal  setting  is  real  time,  that  is,  1  second  of  landing  time  is 
shown  on  the  screen  in  1  second.  The  Slow  and  Very  Slow  options  slow  the 
processing  down  so  that  the  landing  will  appear  to  take  longer  than  the  actual  landing. 
Similarly,  the  Fast  and  Very  Fast  options  will  show  the  landings  at  a  rate  faster  than 
real  time  (if  the  computer  is  fast  enough  to  process  the  data  faster  than  real  time). 


Jumping  to  Sections  of  the  Approach 

The  buttons  shown  below  jump  to  the  beginning  of  the  designated  section  immediately  (once  a  simulation 
has  started).  The  buttons  may  be  used  to  jump  forward  or  backwards  from  the  present  position  in  the 
landing  simulation. 


Waveoff 

The  Waveoff  button  may  be  used  to  signify  a  waveoff. 


Pressing  the  Waveoff  button  causes  the  entire  screen  to  flash  with  a  reddish  background  while  retaining 
all  information.  The  screen  will  look  similar  to  the  following  image  after  the  Waveo  f  f  button  has  been 
pressed. 

Pressing  the  Waveoff  button  again  will  stop  the  waveoff  and  the  flashing. 
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Clear  for  Landing 

The  Clear  for  Landing  button  may  be  used  to  signify  that  the  aircraft  carrier  is  clear  to  accept  a 
landing. 


While  both  the  red  and  the  amber  marks  are  displayed  during  a  landing  simulation. 

f^-ic 

-AR 


m 


Pressing  the  Clear  for  Landing  button  will  replace  the  red  and  amber  marks  with  a  green  mark, 


-AR 

j 


signifying  the  aircraft  carrier  is  clear  to  accept  a  landing. 

Pressing  the  Clear  for  Landing  button  while  the  green  mark  is  displayed  will  replace  it  with  the  red 
and  amber  marks. 


Quitting  the  software 

Either  click  on  the  Exit  button  on  the  main  display  window  or  hit  Esc  on  the  keyboard  to  quit  the 
PADAL  program. 
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SHAI 


Stottler  Henke  Associates,  Inc. 


1660  S.  Amphlett  Blvd,,  Ste.  350 
San  Mateo,  CA  94402 
(650)  655-7242 
(650)  655-7243  (FAX) 
http://www.shai.com 


Certification  of  Technical  Data  Conformity  (May  1987) 

The  Contractor,  Stottler  Henke  Associates,  Inc.,  hereby  certifies  that,  to  the  best  of  its  knowledge 
and  belief,  the  technical  data  delivered  herewith  under  Contract  No.  N68335-98-C-0027  is 
complete,  accurate,  and  complies  with  all  requirements  of  the  contract. 


Signature 


Melissa  Thiemmedh 

Name 

Lead  Accountant 

Title 

3/22/01 _ 

Date 


SHAI 

Artificial  Intelligence  Consulting 


MATERIAL  INSPECTION  AND  RECEIVING  REPORT 


Form  Approved 
OMB  No.  0704-0248 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  30  minutes  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection 
of  information,  including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports 
(0704-0248),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be 
subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  COMPLETED  FORM  TO  EITHER  OF  THESE  ADDRESSES. 

SEND  THIS  FORM  IN  ACCORDANCE  WITH  THE  INSTRUCTIONS  CONTAINED  IN  THE  DFARS,  APPENDIX  F-401. 


1.  PROCUREMENT  INSTRUMENT  IDENTIFICATION 
(CONTRACT)  NO. 

N 68335-98-C-0027 


ORDER  NO. 


9.  PRIME  CONTRACTOR  CODE  |  0K501 

STOTTLER  HENKE  ASSOCIATES,  INC. 
1660  SO.  AMPHLETT  BLVD.,  SUITE  350 
SAN  MATEO,  CA  94402 


6.  INVOICE  NO./DATE 

PAD0035Z  3/31/01 


2.  SHIPMENT  NO. 

3.  DATE  SHIPPED 

4.  B/L 

PAD0035Z 

3/31/01 

TCN 

10.  ADMINISTERED  BY 

DCMC  -  SAN  FRANCISCO 
1265  BORREGAS  AVENUE 
SUNNYVALE,  CA  94089 


code | S0507A 


1 1 .  SHIPPED  FROM  (If  other  than  91  CODE 


SAME  AS  ABOVE 


13.  SHIPPED  TO  CODE  I 

NAVAL  AIR  WARFARE  CENTER  AIRCRAFFT  DIV. 
CONTRACTS  DIVISION(CODE  223000B-2CR) 
HIGHWAY  547,  ATTN:  CINDY  ROLENC 
LAKEHURST,  NJ  08733-5082  _ 


1 6.  STOCK/PART  NO.  DESCRIPTION 

(Indicate  number  of  shipping  containers  -  type  of 
container  -  container  number .1 


15. 

ITEM  NO. 


Data  Item  No.  A013 
PADAL II:  FINAL  REPORT 


21.  CONTRACT  QUALITY  ASSURANCE 
a.  ORIGIN 

CQA  Q  ACCEPTANCE  of  listed  items 
has  been  made  by  me  or  under  my  supervision  and 
they  conform  to  contract,  except  as  noted  herein  or 
on  supporting  documents. 


12.  PAYMENT  WILL  BE  MADE  BY  CODE  |  HQ0339 

DFAS  -  CO-JW 

WEST  ENTITLEMENT  OPERATIONS 
P.O.  BOX  182381 
COLUMBUS,  OH  43218-2381 


14.  MARKED  FOR  CODE  | 

NAVAL  AIR  WARFARE  CENTER  AIRCRAFFT  DIV. 
CONTRACTS  DIVISION(CODE  223000B-2CR) 
HIGHWAY  547,  ATTN:  CINDY  ROLENC 
LAKEHURST,  NJ  08733-5082 _ 


17.  QUANTITY 
SHIP/REC'D* 


20. 

AMOUNT 


b.  DESTINATION 

□  CQA  □  ACCEPTANCE  of  listed  items  has 
been  made  by  me  or  under  my  supervision  and  they 
conform  to  contract,  except  as  noted  herein  or  on 
supporting  documents. 


22.  RECEIVER'S  USE 

Quantities  shown  in  column  17  were  received  in 
apparent  good  condition  except  as  noted. 


GOVERNMENT  REPRESENTATIVE 


GOVERNMENT  REPRESENTATIVE 


TYPED  NAME: 
TITLE: 

MAILING  ADDRESS: 


COMMERCIAL  TELEPHONE 
NUMBER: 


TYPED  NAME: 
TITLE: 

MAILING  ADDRESS: 


COMMERCIAL  TELEPHONE 
NUMBER: 


DATE  RECEIVED 
TYPED  NAME: 
TITLE: 

MAILING  ADDRESS: 


COMMERCIAL  TELEPHONE 
NUMBER: 


GOVERNMENT  REPRESENTATIVE 


*  If  quantity  received  by  the  Government  is  the  same 
as  quantity  shipped,  indicate  by  fXJ  mark;  if  different, 
enter  actual  quantity  received  below  quantity  shipped 
and  encircle. 
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